AAC&U Essential Learning Outcomes and VALUE Rubrics for Sakai

We had a little time in between some projects recently, so we decided to try an in-house project that ended up being something we thought should be out there in the community.

I've been hearing about the AAC&U VALUE project for quite a while now and had been anticipating the project deliverables (a well researched set of rubrics for assessing ePortfolio work) as something we could refer clients to as an excellent example of how to assess student work. It took us a few days to get it done, but the result is something that I think may be useful for the community.

You can download the latest version of the package and read how to install it at OpenEdPractices.org.

An Open Source Implementation of the AAC&U Outomes and Rubrics

It's just good to have an open implementation. Not sure what else to say about that.

A Useful Springboard for Your Implementation

Working in higher education makes me quickly realize that very few institutions want to adopt someone else's idea "as is". However, most institutions don't relish the idea of starting from scratch either, particularly if there IS prior art. Deploying this set of rubrics in a matrix in a live instance of Sakai on your campus could be just the ticket to getting the ball rolling. When you first start up Sakai and look at the portfolio tools, there isn't anything built yet. Making an "institution neutral" experience available to new adopters would be huge step up from the default experience.

The Rubric Renderer

Even if you think these rubrics aren't going to help drive a discussion, having a working set of examples that you can modify is helpful. The default renderers in Sakai don't render anything that looks like a table based rubric. If you follow some simple rules, you will be able to create your own rubrics with the renderer provided AS IS. If you want to change the behavior of the renderer (mostly a bunch of jQuery functions), I've commented the JavaScript in plain english. I'm not a jQuery developer at heart, but it isn't that hard. If you have a question, send me an email..

One challenge we had was to incorporate a large amount of descriptive text provided with each rubric. The obvious place for that information was to place it in the form instructions using the editor to format it just so. However, there is an outstanding bug that sets an upper limit (4000 characters) to instructions in implementations that use Oracle as a backend. To work around this, I escaped the HTML and put it in as the description of the form.

The "Simple Rules" (or "limitations" depending on your POV)

1. You have four "non-zero" choices - The header row of the rubric is created with 4 columns (1-4). A "0" is selected if the evaluator selects none of them. I suppose that it the biggest shortcoming of this render at the moment. I'd like to make this "smart", but am not completely sure how I would do it. I suppose I could count the number of choices for the first criteria and base the width of the table on that. As it is, I knew that I was to have some static content for this application (the designation of benchmark, milestones and capstone), so I knew it wouldn't be completely reusable. It's easy enough to change the renderer to have 3 or 5 columns, but I'd rather it accommodate any number. Tomorrow...there is always tomorrow.
2. Each of your criteria is a string with the number of enumerated values - Each string with enumerated values is going to be rendered as a table in the rubric. I suppose there should be an option for adding other "non-rubric" fields that can accept enumerated values down the road (maybe those without keys?) but I didn't need them for this project.
3. Keys - There is rarely used piece of documentation in form schemas for "sakai.key"'s. (John Ellis implemented it as a way to annotate fields in a subform to display in the main form.) In rubric schema you need to make sure that each criteria has a key and that it is unique. I number the criteria "0", "1", "2", "3', etc... The keys are used in the following ways:
   • The key is used in the ID for the row for the "rubric table" and the ID of the text input for the same criteria. This allows the JavaScript to make sure that when you click a descriptor for criteria "1", that the text value for criteria "1" is changed immediately...and when you type a value in for criteria "1", the right descriptor is highlighted in the table for criteria "1".
   • The "key" is also used to match a criteria to its "rubricWeight" (see below).
   • The sort order of the criteria are determined by their keys.
4. Rubric Weights - This is a new piece of documentation for an element that is ONLY understood by the renderer provided. As we were making the forms, Janice wondered what the "result" of filling out a rubric form would be. AAC&U's documentation didn't provide any clues, but our experience tells us that a final rating should be calculated that is the weighted average of the awarded scores for each criteria.
   
   We could have "hard coded" a weight for each criteria into the renderer, but that would require a new renderer for each new weighting. That wouldn't be very elegant. Instead, some JavaScript variables are populated with the "sakai.rubricWeight" for each criteria. The weight is displayed next to each text input box for each criteria and the final rating (placed into a hidden input) is a weighted average of all of the criteria. Just to be clear, if you have 3 criteria and you want them all to be equally weighted, you would give them all a weight of 0.333333.
   
   This degrades nicely. If you don't use the rubric renderer, the final rating will not be a hidden input that get's automatically populated, but the evaluator will be able to type it in. Unfortuntely, it will make the evaluation prone to error or interpretation by the evaluator.
5. The final rating field is always called "rating" - In the VALUE rubric schema, the element names tend to be verbose and descriptive of what is being evaluated. For example, the "Relationship to Text" criteria element name in the Reading rubric is "relationshipText". The "Connecting, Synthesizing, Transforming" criteria element name in the Creative Thinking rubric is "connectingSynthesizingTransforming . However, in both rubrics there is a "rating" element which will store the weighted average of all of the criteria.
   
6. The comments field is always called "comments" - If you decide to have a comments field and want it below the text input boxes and formatted correctly, you should name it "comments". I elected to make it a simple textarea instead of weighty FCK editor.

Let me know if you find any of this useful or have other ideas for improvement.