Evaluation of MoMA’s online course
Qualitative and Quantiative Analysis
Discover our Story
The Museum of Modern Art (MoMA) is a world famous institution, which offers access and education to the contemporary artworks and art related resources. MoMA supports public involvement in the arts through their educational online courses in Coursera. The courses cover a range of topics from fashion to photography.
Together with my colleagues: Gloriana Amador, Shradha Shree, JiSun Hong and Arielle Bakoss, we have analyzed digital analytics of the “Seeing Through Photographs” course in order to provide valuable suggestions to increase course engagement and determine any structural changes needed.
Duration of the project: 8 weeks
My role: I had leading role in deciding on Methodology. Additionally, I have co-written the report and co-delivered the presentation - both during meeting with MoMA and during Infoshows - event organised by Pratt Institute to present the best student's works during school year.
Research Origin
“Seeing Through Photographs” course is MoMA’s the most popular course on Coursera. However, The research origin lied in the completion rates as “Seeing Through Photographs” had around 154,443 participants commence the course but only 4,767 completed it.
In the end of March 2019, we had a meeting with MoMA's representatives. During it, we have established:
focus of the project: “Seeing Through Photographs” course;
research objectives: to provide the recommendations aiming to improve its completion rate, reduce the drop-outs, increase learners’ engagement;
the scope of the analysed data: Course participants demographics, Course enrollment rate, Course completion data, Course drop-off points and Course engagement rates; Post-course survey, Public reviews (1, 2, 3 and 5 star ratings) and Course Feedback from Coursera.
To ensure the validity of the findings, we have used the triangulation of the data and did the Literature Review and Competitive Analysis. Graph below summarises our sources.
In our analysis, we have decided to focus on the Course feedback file, which consisted of 2178 comments received from learners throughout the duration of the course. These comments, were divided between us and coded using 4 different labels:
Week (Week 1, Week 2; Week 3, Week 4, Week 5, Week 6, General);
Language;
Sentiment (Positive, Negative, Neutral);
Topic (Quizz, Peer Review, Readings, Videos, Photos, Instructors, Interaction, Course)
Comments in a language different than English were translated using Google Translate. We have cross checked our findings with the results of the analysis of Post-survey questionnaire.
The Literature Review, Competitive Review (courses similar to MoMA's course listed on Class Central) and analysis of course Public Reviews gave us background and validated our Findings and Recommendations.
Findings & Recommendations
More than 80% of learners' comments were positive and 14% were considered as neutral. This is really an outstanding result! Lerners praised the quality of the course and professional insights. However, naturally, we still wanted to make the course better. Based on learners' feedback, we have provided 5 Findings & Recommendations.
Language Barriers
Finding 1: Learners look for content in their own language.
This course is a truly global experience - its learners come from different parts of the World and they asked for a high quality video subtitles in their own languages and translations of the readings.
Recommendation 1: Highlight Coursera’s Subtitle Feature
Coursera already offers subtitles in many languages. However, it seems that this feature was often missed.
Learner's comment:
“Es un curso hasta el momento muy bueno, organizado, pero para personas de habla hispana puede resultar un poco complejo, al no contar con subtitulos”
Translation: “It is a very good course, organized, but for Spanish speakers it can be a bit complex, as it does not have subtitles.”
Course Objectives
Finding 2: Participants felt the course objectives were unclear.
Some of the learners thought that MoMA's course will introduce them to the history of the photography or teach them practical photography skills.
Recommendation 2: Clarify course description and objectives in the course introduction.
We have proposed to change the course title to highlight critical thinking, history and theory of photography. Moreover, we suggested to use plain language to describe learning outcomes.
Learner's comment:
“I gained a broader understanding of the history of American photography and some of the forces that has shaped it. Not a new skill.”
Course Accessibility
Finding 3: Increase course accessibility
Learners considered transcript used in videos too small and difficult to follow. Moreover, scanned materials were slightly too academic and difficult too read.
Recommendation 3: Rescan Readings
Rescanned readings and potentially new ones, easier to understand for people who are no longer used to studying will enhance learners' experience.
Learner's comment:
“Better scans, some of them were difficult to read, pdfs should be formatted to fit a single page in full screen. It’s almost impossible to read from a cell phone.”
Assignments & Quizzes
Finding 4: Assignments & quizzes are not serving their function
We have noticed, that questions in quizzes were very detailed and fact based, which did not really teach learners critical thinking. Moreover, assignments were very difficult, which wasn't aligned with the general idea of the course.
Recommendation 4: Refocus quizzes and assignments
We propose to change assignments to enhance critical thinking. Questions could rather ask learners to sum up opinions rather check their memory. Moreover, they can be shorter, which will help learners to stay focus.
Community Building & Support
Finding 5: Participants desired a way to connect with others in the class and feel supported when completing assignments
Learners asked for support in their own language. It was obvious, that they would like to be a part of learning community.
Recommendation 5: Connect and expand course community through social media and alumni as volunteer forum moderators
MoMA is already creating learners community. We have proposed to enhance these efforts and for example promote hashtag #seeingthroughphotographs on social media. This would allow learners to exchange their photos inspired by the course content. It would also incease course accessibility - sometimes it is easier to send a photo than to express yourself in foreign language.
Reflection
In our work, we gave the voice to learners and we worked together to improve the course. "Seeing Through Photographs" thought learners critical thinking. As someone said: "Because it is about understanding photographs, not about taking photographs". Thanks to learners' feedback, we could see that MoMA's course its original purpose.
