New LU grad performs cutting-edge technology research
One of Abby Juozapaitis’ long-term goals is to make a positive impact on women interested in computer science.
The Lakeland University senior’s status as a role model is off to a great start thanks to an impressive Senior Honors Project that researched the impact that robots can have in helping college-aged students with Autism Spectrum Disorder (ASD) to better understand and express emotions.
Juozapaitis, who is graduating this month with a double major in computer science and Spanish and a minor in mathematics, spent much of her 3.5 years of research working with Pepper, a 4-foot-tall humanoid robot capable of recognizing human emotion. Lakeland purchased it in the fall of 2019 thanks to a grant from the Ruth St. John & John Dunham West Foundation.
The idea for this project came from Lakeland Associate Professor of Computer Science Cindy Lindstrom as part of a directed reading course, and it happened to be around the time LU purchased Pepper. The directed reading helped Juozapaitis understand ASD, and the acquisition of Pepper fueled her passion for the potential to positively impact students.
“Once I realized that the robots we had in the tech lab could make an impact on people with ASD and on campus, I became more intrigued and researched everything I could think of on this subject,” said Juozapaitis, who came to Lakeland from North Prairie, Wis., and Mukwonago High School.
After watching former LU classmates (now Lakeland grads) Nick and Dan Koerber award-winning work with Lakeland’s NAO robots and experimenting with her own programs, Juozapaitis took her learning to a new level with Pepper. She designed custom applications/programs so Pepper can recognize human emotions.
“Pepper is a whole different ball game,” Juozapaitis said. “She’s capable of a lot more than the NAO robots in terms of mobility, types of human interaction, modules she can use and machine learning. I got her to detect smiling, frowning and neutral expression, but I think we could get her to recognize more complex expressions such as surprise, anger and boredom.”
Lindstrom said Lakeland could find no research that’s been done that used a robot like Pepper in such a capacity.
“This could have significant impact,” Lindstrom said. “Abby's level of work was on par, or even above, what one might see at a larger, research institution. This is very unusual. I have talked with our robot provider and they are amazed that we are able to design such unique, custom applications for our NAO robots and now with Pepper.”
Jessica Kalmar, director of the LU Honors Program and an associate professor of psychology, said Juozapaitis’ project is a stellar example of what honors students can accomplish with the opportunity to conduct independent, interdisciplinary research.
“The sophistication she displayed when incorporating her programming skills with knowledge she amassed about the educational challenges facing students on the autism spectrum in collaboration with Latin American scholars was at the level of a graduate student – or beyond,” Kalmar said. “Students in the Psychology Laboratory hope to work with Abby to continue to carry this research forward.”
Juozapaitis spent about 10 hours or more a week in the tech lab writing code and experimenting with Pepper, along with time organizing a massive amount of research. Her final project includes a final presentation (which you can watch here) with video of her interactions with Pepper, a 37-page research paper and three different programs written for use with Pepper.
Juozapaitis’ honors project included a second directed reading conducted entirely in Spanish with LU Spanish Instructor Reina Drake. Juozapaitis connected with some of the authors who wrote articles or journals she read, and she interviewed a special education teacher about technology she uses with students she teaches who are on the spectrum. That research helped her map out a plan for how to work with Pepper.
Pepper is most often used as a concierge robot to take restaurant orders or give customers directions in an airport or retail settings. Juozapaitis said there are numerous potential opportunities for Lakeland students to work with Pepper, including getting her to better understand whether a user is looking at her and further developing her programs so Pepper can recognize more emotions.
“It would be really cool if someone programmed her to give students tours of the Chase Science Center,” Juozapaitis said. “There could definitely be more programs written to help people on the autism spectrum in college, but there's a whole other world of possibilities.
“Hopefully my research inspires future Lakeland students. I also have hope that some of my programs could be used as examples or built upon in programming classes. Even though I’m graduating, it would be cool to still work with my professors and with the robots in the lab.”
Lindstrom said Juozapaitis may not fully appreciate the impact she’s already had on other young women in Lakeland’s computer science program.
“Abby was the first female president of our Technology Club, and next year, Korryn Clint will be the second,” Lindstrom said. “In fact, Korryn is also interested in using the robots for her honors project. I have a feeling Korryn will take the reins and continue where Abby left off.”
Lindstrom and Drake both played key roles for Juozapaitis by challenging her to find more information on a variety of topics relating to ASD and robots. “Dr. Drake conducted my entire directed reading in Spanish, which really helped me not only gain knowledge in my project topic, but helped me develop my Spanish speaking and reading skills,” Juozapaitis said. “It also was a good experience finding research from Latin American countries and seeing this project in a global view.”
Another important resource was her academic advisor, Professor of Mathematics Chia-Chin (Cristi) Chang, who encouraged Juozapaitis to take programming electives for the computer science major.
“Without her Python class and both her and Cindy’s assistance at Tech Club meetings, I probably wouldn’t have developed an interest in the robots,” said Juozapaitis, who was a member of the LU women’s tennis team, the Phi Delta Omega sorority and served as a Campus Ambassador.
“The majority of what I learned specific to the robots resulted in hanging out with all the guys in the tech lab, eating pizza and trying to get the robots to do different tasks.”
Lindstrom said the support of the West Foundation in funding Lakeland’s purchase of the humanoid robots has given a small school like Lakeland an edge over some much larger higher education peers.
“Because we are small, I can work directly with the students on these projects, where larger universities rely on graduate students to work with underclass students,” Lindstrom said. “Our Lakeland community can quickly see the impact the robots have on the program, and now everyone will start to see the fruits of this research.
“There is much interest in the community for helping neurodiverse students, and we will continue to seek out grants and donors to sponsor our students and their research into how technology can be used to help these students. Abby is one of the trailblazers that’s leading the way.”
Juozapaitis has accepted a job as an associate IT applications consultant with WEC Energy Group (We Energies) on the GIS (Geographic Information Systems) Team out of Milwaukee. She interned with the company in IT applications for almost two years. In this full-time role, she will provide support for various geographic and mapping software/applications used throughout the company, and develop new applications for use throughout the company.
Long term, she has thoughts of being an IT project manager and after spending a few years in the industry she will consider returning to school to earn a master’s degree.
“I’ve been challenged, encouraged and given opportunities here that I don’t think I would have received attending a different university,” Juozapaitis said. “I have very close relationships with Dr. Lindstrom, Dr. Drake and Dr. Chang. My Lakeland experience has truly shaped who I am and allowed me to make and achieve goals I never thought would be possible.”
