Blackboard Analytics for Blackboard Learn Customer Limited Field Trial Panel: Early Findings and Insights
Eric Kunnen, Director of Distance Learning and Instructional Technologies, Grand Rapids Community College
John Fritz, Assistant VP, Instructional Technology and New Media, University of Maryland, Baltimore County
Celeste Schwartz, Vice President for Information Technology, Montgomery County Community College
There are three pillars behind Blackboard Analytics for Learn:
- Improve Student Success
- Optimize Blended & Online Instruction
- Leverage LMS Investments
In a nutshell, data is being taken from where Blackboard Learn and Student Information Systems and put into a transformation layer. Then it is stored in a data warehouse and finally end users can get information from the system using dashboards, reports, and dynamic analysis.
UMBC Background
- Founded in 1966
- A research extensive university based on Carnegie classifications
- Fall 2011 – 13,199 students (10,573 undergrad, 2,626 grad) and 1,186 staff
- Selected brags: #1 “Up and Coming National Unversity” U.S. News American’s Best Collegs, three years running, 2011, 2010, 2009, 1st in Undergrad Chemistry Degrees awarded to African Americans, One of 50 Best Colleges for Women, 7-time National College Chess Champions
- Began using Bb in Spring 2000
- Currently on Bb 9.1 SP 6 with plans to upgrade in the Winter to the latest version
Adoption stats
- 95% of all students
- 75% of all instructors
- 65% of all courses
- 350 communities
Analytics Initiatives
- 2007 – Started publishing Public Reports http://www.umbc.edu/oit/newmedia/blackboard/stats/
- 2008 – Check my Activity for Students
- 2009 – Code release at BbWorld 2009
- 2010 – Adopted iStrategy for analysis of all Bb courses
- 2011 – BbA4L Limited Field Trial
Support Staff
- 2 FTE (Administrator and Support)
- 1 Server Admin
- When they started in 2000 they had the same number of support staff that they do today.
GRCC Background
- Grand Rapids, MI
- Enrollment of 17,000+
- 200-300 Full Time Faculty and 600-800 Adjunct Faculty
Analytics Projects:
- New Data Warehousing Initiative
- BbStats & Project ASTRO Building Blocks
- Bb Analytics & Analytics for Learn Limited Field Trial Implementation
Blackboard Environment
- Bb 9.1 SP 6 with plan to implement SP 9 in August
- Currently in a Summer Pilot with SP 8
- Bb Learn, Community, Content, Connect, Collaborate (Bb IM, Voice Tools), Transact, NBC Learn, and Starfish Early Alert
- PeopleSoft is the Student Information System
MCCC Background
- More than 34,000 annual unduplicated credit and non-credit students
- Over 15,000 fall 2012 students expected, slightly under 15,000 in fall 2011
Campuses
- Blue Bell (an affluent area) and Pottstown (a “river town”, industrial area)
- Online
- Culinary School (2013)
- Suburban Philadelphia location in County with 800,000 residents
- Highly competitive higher education marketplace – there are over 8,000 universities within one hour drive from the Blue Bell campus
Technology Environment
- Bb Analytics for Learning Limited Field Trial, pilot group has recommended to move forward and purchase BbA4L
- Bb Analytics Student Module
- Blackboard 9.1 SP 7
- College ERP: Ellucian Colleague
Audience Questions
What were your key findings in the field trial? Successes, new information, and areas you are hoping for improvement in?
EK: One of the most amazing thing we found being part of the field trial was seeing the data that is available. The SIS data combined with the Learn data gives you much more insight. Demographical data about your students using the tools in the system and being able to slice that among different departments and instructors gives you a great amount of potential to view trends in data. One of the biggest challenges with access to this wealth of data, it becomes overwhelming very quickly. The dashboards were one of the most important pieces, the ability to go into the system to get an at a glace view of the pulse of the system is very beneficial. Plus, by viewing this information on a dashboard you can quickly copy and paste the information or let academic leaders know that they can view the dashboard at anytime to gather this data.
CS: The thing that was most powerful for Celeste was what the students can see. When you get down to data that you can give to students at the student level, access statistics and grades with side-by-side comparisons of how other students in the same course are working. How much time are A, B, C, and D level students spending in the course as compared to “me”. This may help faculty identify content that is not working, and areas that should be reworked. It is a huge driver for student success and retention.
JF: Trying to come to agreements on terminology like what is a course. The data you can get helps you get to the culture questions quicker. You can get the data very quickly, but once you have that data, what do you do with it? You have to look at the same data with at least one other person because it is amazing how differently you view the data and the variety of nuances that arise. One of the biggest perspectives available is the ability to view how much content is in the courses and how many students are actually accessing that content. There is great potential for faculty development, course design, and student success.
With the awareness of all the data and the power of it, what are some of the other questions you are now able to ask and answer that you were not able to previously.
JF: I am a big believer in the hidden tool of adaptive release. When we tried to do some analytics on our own, I was starting to hear more faculty using adaptive release. Mark put together a report that shows all of the courses that use adaptive release versus those who do not and there is a 10-15% increase in students accessing content in courses that use adaptive release. This was data that would have been impossible to find before using A4L.
If faculty could just see that they are spending a lot of time on a course that students are not using. If you faculty can see that their colleagues are spending less time on developing their courses and having their students access their materials more, than why wouldn’t they rethink how they are designing their course and go talk to those colleagues who are spending less time on development.
EK: At GRCC, we are at the cusp of being able to ask questions surrounding instructor presence in courses and how important that is to students. How do we better train our faculty? Who are our faculty who are using the various tools? There is an amazing amount of potential combining the data that is currently in the student information system with the data in Learn.
Global level outcome reports for general learner outcomes was painful before, and with this tool will be much easier for us. You can use your time to ask questions, process the data, look at trending instead of using all your time gathering the data.
CS: We have very few tools in our tool kit that allow us to merge student information from an SIS with the LMS data. This tool has allowed us to co-mingle those things and be able to look at the data in a very different way. The real power is going to be at that individual faculty and student level.
What documentation is available that will help sell this tool to faculty? Both for those that are interested in using learning analytics, but also those who are resistant?
All reports have a link to the documentation within the tool. Outside of the tool, Blackboard has a series of other documents for different roles. Some institutions have also begun to create their own more custom documentation.
JF: This is a powerful tool which will take some time. You have to use it a little bit every day. As you begin to operationalize and use it on a daily basis, you will get much better using the tool. Although right now I spend 30-40% of my time using this tool right now, however, I do not have to use anyone else’s time to get the data.
EK: The time investment depends on your role. There are reports prepared specifically for deans. Your institution’s set-up will also determine how much time you will spend on the tool. It takes a commitment that involves the entire campus, not just one particular office. It is a collective campus need that requires inclusion from all parties.
For more information on Blackboard Analytics for Learning visit: http://www.blackboard.com/Platforms/Analytics/Products/Blackboard-Analytics-for-Learn.aspx
GRCC BbWorld Conference Blog 