Thomas Hart built his house near the bluff overlooking the flood plain of Black Creek around 1820 with timber cut on the property. The dwelling is typical of farmhouses of the era; one room deep with a central hall upstairs and down. Each of the four original rooms contains paneled wainscoting and mantels carved by the young Hart.
The plantation grew to 1,223 acres of cotton, tobacco and other crops, as well as wild lands. Thomas Hart was the first postmaster of the area, a merchant, justice of the peace and a captain of a local militia company. The town became known as "Hartville" or "Hartsville" as early as 1837. The area was soon to grow even faster with the arrival of more settlers, including Major James Lide Coker. After Captain Hart’s death, his wife continued to live on the property until 1859.
Around the turn of the century, the property changed hands many times, eventually passing to the wife of David Robert Coker. Mrs. Coker proceeded to turn the neglected site into a scenic attraction. Azaleas, camellias, wisteria, tea-olives and others were added along the trails. Old railroad ties were added to trails, and a pond was dug. The result was named "Kalmia" for the plentiful laurel Kalmia latifolia. Mrs. Coker also added four rooms to the house, but did not alter the integrity of the structure.
Since 1935, Kalmia Gardens has been open to the public free of charge from dawn until dusk, every day of the year. In 1965, Mrs. Coker gave Kalmia Gardens to Coker College. In 1996 the Hart House was renovated, and a semi-formal Garden and a Sensory Garden were established. Kalmia Gardens of Coker College is now an outdoor classroom and public garden, hosting many field trips from area schools as well as public programs, tours, symposia, plant sales, festivals and garden parties.