James Oliver Morton was born December 20, 1820, in Screven County, GA., one of the children of Silas Morton and his wife Sabrina Archer, and by ancestry represented one of the oldest American families. He was descended from George Morton who landed at Plymouth in 1622 and joined the Pilgrim Fathers. Generations later, a descendant, Oliver Morton, came from Massachusetts, via North Carolina, to Georgia, with a son Silas Morton, the father of James Oliver Morton.
James Morton attended the local schools in Screven County, also Chatham Academy in Savannah. He was married in August, 1843, to Miss Sarah Young of Bulloch County, Georgia, daughter of James and Lavinia Young; she proved a wonderful helpmate, and by her sympathetic understanding and assistance, contributed very materially to his success. Immediately after their marriage they came to South Georgia and located in what is now Brooks County, where he lived until his death in September, 1911, in the 91st year of his life. Mrs. Morton survived him about five years. Both were buried in West End Cemetery, Quitman. They had two children: a son, Simeon L., who at the beginning of the War Between the States joined the Savannah Volunteer Guards and was killed in the last fighting around Richmond. A daughter, Lavinia Calhoun, married Henry G. Turner.
When Mr. Morton and his bride came to South Georgia much of this section consisted of pine forests and scattered swamps, making it necessary to clear the land for his farm and home. By good management, energy and economy he succeeded in acquiring a modest competency in a comparatively short time, and was then able to and did devote much of his time and attention to the civic affairs and the development of his home town and county. Having demonstrated his ability and integrity he was named as one of the first Justices of the first Inferior Court of Brooks County. The county was created in 1858 and its affairs were managed by members of that court. After serving as a member of the Inferior Court he was known as Judge Morton. He also served as a member and chairman of the County Commissioners, 1881-1885. Brooks County and Quitman, the county-seat, having been established, it was necessary to have banking facilities to handle the business and fiscal affairs of both, so the Bank of Quitman was organized. This bank still has the same name and is considered one of the safest banking institutions of the State. Judge Morton was one of the founders of the bank and was made president shortly after it was organized and served in this capacity until his death. He was active in the management of its affairs, visiting the bank every day by horse and buggy. His last trip was made a few days before his death. During his long and useful life, by reason of his good business judgment and close attention to his affairs, he accumulated a very creditable estate.