University of Delaware MIS professor Mark Serva knows that data comes in all shapes and sizes: linear and nonlinear, relational and non-relational. He also understands that developing ways to analyze this data is key for financial companies to make critical business decisions. This means that future hires who have knowledge and skills in these areas will be in great demand.
So last spring, when undergraduate MIS students Emma Lavelle, class of 2018, and Julia Forster, class of 2019, approached Serva separately about independent study opportunities, he knew exactly what type of project he wanted them to tackle.
Serva was returning from a sabbatical with a major financial institution where he worked with HADOOP, a cutting-edge file system that allows for the rapid analysis of financial data. The HADOOP file system is able to pull in large quantities of data quickly and has become a standard platform for Big Data technologies. This inspired Serva to rethink the MIS curriculum at UD’s Lerner College.
Lavelle and Forster, who both had Serva as their adviser, were looking for additional opportunities to help them stand out to employers once they graduate. Serva had them team up as part of their study to introduce data analysis with HADOOP and cloud computing into the MISY330 database class by the end of the spring semester.
“The MISY classes fulfill my interest in technology,” Lavelle said. “There are so many opportunities to learn and build up your skill set.”
“I’m interested in cybersecurity and project management,” added Forster, who is also a double major in operations management. “Our MIS courses cover a huge range of topics in the technology sector. I am happy that my courses are so relevant for my future, and I am happy to be an advocate for women in the technology field.”
“We can no longer think of data as simply relational; it is coming at us in many different formats,” Serva said. “With the growing importance of data science, HADOOP allows for accessing data in more flexible formats than relational databases. Technically, HADOOP is a file system, not a database. But its speed and flexibility has made it one of the most important data storage technologies in data science.
“In order for independent studies to be successful, there has to be a specific task in mind,” Serva continued. “Both Emma and Julia are talented and quick learning students. I felt confident that they could turn this independent study into something that could contribute to the revision of one of our core MIS courses.”
Bringing HADOOP To MISY330
Lavelle and Forster were tasked with researching and understanding how to configure a server that could support HADOOP and could be accessed by 80 UD students in the class. The team would also need to help students understand Pig Latin, a HADOOP-based language that allows users to generate data queries. The class would be analyzing data sets in multiple formats, including relational and non-relational databases.
“HADOOP is a complex system with many potential uses; you can essentially do anything you want with data,” Lavelle said. “This was not a ‘nice and easy’ independent study. It was wonderful to be able to learn together, as well as to have support from the accounting and MIS department.”
“The project was daunting,” Forster added. “HADOOP did not come with instructions; it was unclear even how to start. Emma and I had to organize the information and Professor Serva and the MISY330 students had to trust us.”
Lavelle and Forster used peer-to-peer learning to develop their framework for building out course materials for the HADOOP instance that they were bringing into the lab, while checking in with Serva along the way. Because so many students would be accessing and running data concurrently, Lavelle and Forster first tested the system before building lecture materials around it and rolling it out. Using load testing and query generation in Pig Latin with four of their classmates, they quickly determined that the server would not be adequate to handle the necessary requests.
“We had to start over and come up with another solution,” Lavelle said.
While Lavelle researched other server configurations, she and Forster learned that UD was exploring cloud computing. With help from Samantha Newell from the Lerner IT department, they settled on Amazon Web Services (AWS), a preeminent provider of cloud computing. The team contacted AWS and worked out an arrangement to allow UD students to use HADOOP with AWS as the server provider.
“In hindsight, this development was actually beneficial because it opened us up to our ultimate solution, a cloud-based server,” Serva said.
For Lavelle and Forster, the process challenged them to see how AWS, HADOOP and Pig Latin would work together in the cloud before the end of the semester. As Lavelle said, “The best way to learn a system is hands-on, through direct experience.”
Learning Pig Latin and Other Outcomes
The next step was to develop classroom resources that helped students understand Pig Latin.
“I liked the data analysis and project management, and Emma liked the programming part, so we balanced each other out,” Forster said.
During two MISY330 class meetings, Lavelle and Forster taught the students HADOOP and Pig Latin and received feedback on where to make improvements for future semesters. The class performed well on the exercises.
Serva, Lavelle and Forster said that they were pleased with the project’s outcome. Serva was able to apply Lavalle and Forster’s materials to the fall class and all classes going forward.
Lavelle is now finishing up a Global Enterprise Technology (GET) Immersion Internship this fall with Deloitte, and plans to graduate in December. She said that she was able to apply the time management and project management skills she learned in the spring to her internship experience.
Forster participated in the EY Risk Advisory program, and said that she found the experience bringing AWS into the project and analyzing the data is something that will apply to her future career in operations management.
Both students said that they would recommend an independent study with faculty to any student looking to challenge themselves. “It was a memorable experience,” Forster said.
Lavelle and Forster also encourage faculty to create more learning opportunities like this for their students. “Not everyone has an adviser like Dr. Serva,” said Lavelle. “But we had a great opportunity; it was awesome. Other students should be able to have an experience like this.”