Celeste felt inadequate as an early teacher and wanted to learn more about the profession. So she read books and finally attended classes at Roanoke Female College and summer seminars at the University of Virginia, but she hoped for an "actual" college diploma. In 1884, when Celeste was 31 she enrolled as a student at the State Female Normal School in Farmville (now Longwood University), the first state school founded to educate women teachers.
In 1893, she became chair of the Mathematics Department at Randolph Macon Women's College, where she also taught psychology. While there, she helped establish the first psychology laboratory in the South. During the summers, Celeste took summer classes at Cornell University and received special permission to take correspondence classes. In 1896, she earned a Bachelors Degree from Cornell University in Philosophy.
Celeste felt strongly about making higher education accessible to women, she wrote articles and urged female seminaries to become colleges and became the first President of the Southern Association of College Women. She also fought to reform public education and in 1912, became the Rural School Supervisor for North Georgia, traveling by coach and buggy to over 2,400 rural schools to teach and train teachers. Celeste died in 1918 and is buried in Georgia; her gravestone includes the inscription "Georgia's Greatest Women"