The Brown Family of Dry Fork

During the War Between the States

I live on part of the old Benjamin “Ben” Brown farm. This property has been in the Brown family since Ben obtained it through land patents in 1845. Ben was a farmer, operated a small grain mill, and a part time surveyor (I am a surveyor as well). His farm was on Dry Fork Creek of the North Fork of the Kentucky River about four miles from Whitesburg. He was the son of John Quincy Brown (first school teacher in Letcher County and a veteran of the War of 1812) and Elizabeth Caudill (daughter of Stephen and Sarah Caudill, Stephen was a Revolutionary War soldier). During the War Between the States, the Brown family was pro-union while most of their neighbors supported the southern cause. Ben was a member of the Union home guard as were some of his brothers and a son.

Throughout the war, no major battles involving thousands of men were fought in Eastern Kentucky but constant fighting, small battles, skirmishing and ambushes occurred daily in the isolated mountains. These small-scaled shootings usually pitted family against family and neighbor against neighbor. It truly was a “Brother against Brother” war for the mountain residents of southeastern Kentucky. It was not long before families from both sides were constantly accusing each other of bushwhacking (firing on a person while hiding in the bushes, ambush style). Ironically, part of the legendary “Rebel Trace” runs through Dry Fork, crossing both the Cornett and Brown property. Up until the war, both families worked together to make this section of the road passable by wagon, all the way from Dry Fork to Whitesburg.

The local Confederate regiment, the 13th Kentucky Cavalry (originally this unit was organized as the 10th Kentucky Mounted Rifles), was stationed in a large bottom west of Whitesburg that was owned by John Caudill. Colonel Ben Caudill, their commanding officer, was under increasing pressure to arrest any Union people in the area that were suspected of harassing Confederate sympathizers. One of the families that he was to punish was his first cousin, Ben Brown’s (Ben Caudill’s father, John Caudill, and Ben Brown’s mother, Elizabeth Caudill Brown, were brother and sister). This put Colonel Caudill in a predicament, as he knew that these arrest usually resulted in bloodshed. To try to avoid having his own kinfolk harmed, he decided to send Fifth Sergeant John B. Cornett. John was a nephew of Ben Brown’s, as his mother, Sally Brown Cornett, was a sister to Ben. John’s father was Joseph E. Cornett whom himself had been in the 5th Kentucky Infantry for a short time. Joseph Cornett had also been the second judge of Letcher County and was the Letcher County Surveyor for many years (his brother-in-law, Benjamin Brown, helped him survey dozens of land patents).

Having delayed the eventual visit as long as possible, Colonel Ben Caudill ordered Sergeant Cornett to arrest the Browns on Dry Fork Creek. On a cold, January morning in 1865, the sergeant and part of his company of the 13th Kentucky Cavalry, began to ride to Dry Fork (a local resident had informed the colonel that all of the Brown boys were gathered at Ben’s cabin) to arrest his cousin, James Brown and his uncles Jesse Marlin Brown, William Brown and Hiram Brown, (these three were brothers to John Cornett’s mother, Sally “Sarah” Brown Cornett and all had been in the Union Army, first the Harlan Battalion and later the 14th Kentucky Infantry) and possibly some of the other Brown boys.

Jesse Brown was no stranger to the 13th Kentucky as he had been captured by members of that regiment while serving in the Harlan Battalion. He was later exchanged for his first cousin, Samuel Cornett (son of Joseph and Sally Brown Cornett), a member of the 13th Kentucky, whom had been captured by the Harlan Battalion. Jesse’s brother, William “Billy” Brown, had also been in the Harlan Battalion and had a finger shot off while fighting against the 13th Kentucky in the Battle of Whitesburg. Whether any of the Brown boys were actually bushwhackers is unknown but could easily have been as they and their families were constantly being robbed and attacked by southern sympathizing neighbors.

Sergeant John Cornett probably was not very sympathetic to any Union sympathizers, as a Union cavalry patrol had recently visited his family’s house and turned over their bee gums, took their livestock and other food supplies. When asked what were they supposed to feed their children, the commanding officer replied, “ I hope the bees sting you all to death and you can all eat browse for all I care”. Sadly, these deprivations occurred to families on both sides.

As the Rebel cavalry were riding up Dry Fork Creek, someone observed them and ran to Ben’s cabin with the news. The Brown boys had been expecting the visit and hid in the hills above the house. Upon arriving at their destination, the Confederates were furious that they had missed the boys after riding so far to capture them. Suspecting that the Brown boys were close by, Sergeant Cornett, placed Ben under arrest as a hostage. In a loud voice, he informed Ben’s wife, Mary, that if the Brown boys wanted to see Ben again, they should come to Whitesburg and turn themselves in.

The Brown boys saw what happened and cut through the hills in an attempt to cut the Rebels off before they could get to Whitesburg. While riding down Dry Fork Creek, Ben overheard some of the Rebels say that they planned on hanging the Brown boys when captured without the benefit of a trial. Ben hoped that his nephew, John Cornett, would not let that happen but he could not be sure of it. As the Rebel patrol rode toward Whitesburg, they stopped at John’s father, Joseph E. Cornetts’ cabin on Dry Fork, which was just a mile downstream from Ben’s farm. Joseph and Sally begged the Confederate soldiers to let Ben go. Reluctantly, Sergeant Cornett said he could not go against orders and continued toward Whitesburg with his prisoner, passing by his own house at the mouth of Little Dry Fork. At a spring on the upper side of the road leading to Whitesburg just past John’s house, the Rebel patrol stopped to get a drink of water. The Brown boys had easily gotten to the spring first and were hidden in the woods above the spring. When the soldiers of the Rebel patrol dismounted to obtain a drink of water, the hidden Brown boys informed them to release Ben. The boys in gray refused and said they would shoot Ben first if the Brown boys didn’t surrender. Ben hollered and told the boys not to surrender as they would be hung, and spurred his horse in an attempt to escape. The Rebel soldiers then shot Ben (John Cornett later swore that he didn’t fire his weapon). Seeing his uncle shot took the fight out of Sergeant Cornett and immediately ordered his troops back to Whitesburg.

After the Confederates had left, the Brown boys took Ben Brown to Joseph and Sally’s cabin where he died in front of their fireplace on the floor. The bloodstains were always there after that and were still there when the cabin was destroyed by fire in the 1960’s. My mother, Fern Caudill Brown, (great-great-granddaughter of Joseph and Sally Brown Cornett and great-granddaughter of John B. and Elizabeth Hays Cornett) remembered seeing the stains as a little girl.

The grieving widow and her sons buried Ben on the hill not far from his home on Dry Fork. To mark the corners of the grave, they planted small cedar trees. (The Letcher County Historical Society placed a monument at his grave several years ago). After this incident, the Brown boys realized that it would not be safe for them to remain at home any longer and rejoined the Union army (Three Forks Battalion).

A few years after the war, part of Ben’s farm was sold due to a debt incurred by Ben prior to the war. David Tyree bought this portion of the property at an auction conducted by the sheriff of Letcher County. His daughter, Rachel, married Ben’s son, Elihu. Elihu and Rachel Brown lived on Dry Fork on the part of his father’s land that he inherited. My aunt, Lillian Brown Boggs, and her husband, Ray Boggs, now own most of the property inherited by Elihu. My great-great-grandfather, William Ransom Brown (Ben’s youngest son), inherited the part of the farm that I live on.

The killing of Ben caused hard feelings between the Browns and the Cornetts for many years to come. My father and mother’s generation saw the warming of the relations between the two families (luckily for me). The Browns blamed the Cornetts for Ben’s death and the Cornetts blamed the Browns for ambushing Joseph E. Cornett and partially crippling him. My grandfather, William Henry Brown (great-grandson of Benjamin Brown), fell in love with Alberta Cornett, great-granddaughter of Joseph E. Cornett and granddaughter of Sergeant John B. Cornett. They planned to sneak off and get married, as neither family would allow the marriage to take place. Alberta’s family found out the details of the planned elopement and had a friend of Alberta’s write a letter to Henry. This girl pretended to be Alberta. The letter stated that Alberta did not love Henry anymore and did not ever want to see him again, that she had met someone else. Henry’s father, Shade Brown, was a teacher and had been trying to get Henry to go to college. Henry felt it was a good time to leave town and went to Barboursville, Kentucky, to attend Union College. While there, he got into a very bad fight with another student and seriously injured the boy. At the time, Henry thought he had killed the boy and fled school, joining the Army.

When his parents found Henry, they convinced the Army to release him. Upon returning to Dry Fork, he discovered that Alberta had married R.B. Caudill. Their daughter, Fern Caudill Brown, is my mother. Henry then met and married Florence Kathleen Richmond from Colly Creek. Her grandfather, Jonathan Richmond, was a lieutenant in the 64th Virginia Cavalry, her other grandfather, Henry Powell Collins, was a private in the 14th Kentucky Cavalry U.S.A. Henry and Florence’s son, Glenn Brown, is my father. My grandfather did not learn the truth about the forged letter until many years later. Strange how things turn out isn’t it?

Another strange twist to Benjamin Brown’s story involved his relatives from Ashe County, North Carolina. When Ben’s father, John Quincy Brown, left North Carolina to move to Letcher County in 1815, some of his brothers stayed in Ashe County. John’s brother, George Brown, (after the war, George moved to Johnson County, Kentucky) had seven sons in the Confederate Army, all in the 21st Virginia Cavalry. For about a year, this regiment was stationed at Saltville, Virginia, and fought in a skirmish in Harlan County. The North Carolina Brown boys came to visit their Kentucky cousins when they were stationed close to Letcher County. One of George’s sons was also named Benjamin Brown, and was killed near the end of the war at Farmville, Virginia. Ironically, the war claimed both Benjamin Browns, one for the Union, one for the Confederacy.

Richard Brown