Writing and Exporting Feedback

On the TBX Forum, I was asked a question about the rubric I built a few years ago. I don’t use it anymore because I built it to grade in a style I don’t like or believe in. (Which was dumb on my part.) But I suggested I might make a video showing what I’m planning on using this coming term.

This seemed possible because I was thinking through how to cope with the fact that a lot of the brief in-person feedback I give throughout a term will, this term, necessarily happen online and in writing. What I imagined doing was sitting down to put my new system together and just recording what I did as I worked. When I was done, I expected I’d be able to cut out the lulls and moments of despair and post what was left.

The result I arrived at are the two new videos I’m posting today. Together they record me beginning with a fresh starter file and from there creating a form for offering quick feedback on small assignments that exports as a custom text layout. My total work time was about an hour and a half, but I’ve managed to cut that down to two twenty minute videos.

These will be the last videos for awhile: Day One looms, and the pressure is on.

Tinderbox Videos

It’s the time of year where I’m preparing courses for the Fall, and this year, I decided that I would make some videos demonstrating some of how I use Tinderbox to do that work. Two of these are now live. I’m hoping to make and post a few others in the coming days and weeks.

These videos are something I’ve imagined doing in a vague way for awhile and so they have been a long time coming. But in another sense, these videos are very much a product of the pandemic.

Classes are online for the Fall, and this has both increased the complexity of my preparations and forced me to do much more of them before classes begin. There seems to be so much to do and so little time to do it that it feels overwhelming. Making these videos offered me a chance to step back from that work and reflect on how I was getting started.

One important point: these videos are not instructional “how to” clips. Instead, they are a couple of brief looks at a few of the ways I go about my business.

Enjoy.

Some Final Meta-blogging

Now that I’ve sorted out how to style a personal theme, there are really only two things that annoy me about this WordPress blog, neither of which I can do much about: speed and security.

Speed: the flat HTML site spit out by TBX was stupid fast. Clicking the homepage link meant seeing the page before I was ready to take it in. There was absolutely no noticeable delay. WordPress is not like this at all. Sitting on the same server, it loads slowly enough that I regularly have to check the progress bar to convince myself the page is on its way. And this is with the default theme, a homepage with no images and only three basic plug-ins. It’s excruciating.

Security: Those plug-ins? They enable features like ssl and failed-login locks because security is a problem with WordPress. This morning alone a steady stream of automated, regularly paced (but failed) login attempts were logged at the site. This was basically a non-issue with the flat HTML site I worked up because there were no log-ins, no databases, no php. It was just HTML text, elegant, beautiful and solid. I miss that. Maybe (probably) I don’t understand what these log-in attempts are and maybe they’re benign, but it feels like keeping things locked down here is a pain.

Worn In

I have a pair of leather shoes that I bought in 1997. I was a student, had no money, and they were high-quality enough to cost more than I could afford. But I bought them, and now, twenty-two years later I still have them, still wear them and they are the most comfortable shoes I own.

This blog is a lot like those shoes: I’ve broken it in enough for it to be hard to give up. Even if it’s rough and worn, it fits.

Or to use a different analogy: Jack might tell Innes “I wish I could quit you,” but he doesn’t. And I guess it’s the same with me and this damned WordPress blog.

A Theme of One’s Own

This morning I finally sat down and figured out the odd element tags in WordPress and wrote up the additional css I needed to make the default theme match the solarized theme I’d built for my MOPI notes. The basics were easy, but filing off the rough edges took some digging around in the main stylesheet.

I like the results and feel like I’ve got a space I can post in again.

(Aesthetics shouldn’t matter this much but they do. They just do.)

TBX on the Go?

Tinderbox only runs on macOS. There’s no iOS version. Generally, this is fine because if I’m going to work, I’m usually going to be working at my computer. The one major exception to this is when I’m drafting. This is tough work and I need to be able to move from computer to paper and back again, and I need to be able to decide that “this isn’t working” and to pick everything up and go to a cafe.

How’s this supposed to work if the research materials I need to reference while I’m writing are bound to my desktop? It occurred to me today that the obvious answer is HTML export.

When I’m writing, I never really need to change my TBX file. In fact, in a perfect world I would probably be prevented from making changes to it because it’s way too easy to sit down to write only to discover, an hour later, that I’ve produced a beautiful new map rather than 500 words. But here’s the thing: if all I want to do is to be able to read my notes, then it should be easy enough to just export and post them onto my server. I don’t even have to worry about whether the pages look good, right? They just need to work.

So to try this idea out, I copied the CSS and the main page template used in this blog into my research file. I wanted a template for each prototype (I’m using them to ID different kinds of materials) so that I could make sure that the appropriate information for each note would appear on the web pages that exported. To avoid making stupid HTML errors (I’m not a computer tech), I just kept duplicating and editing the main template I’d copied over from this blog. As I edited, I expected to be pulling info from attributes with value() statements, but I soon realized that, because I’ve been relying on links and link actions to organize my research ever since the new link pane was introduced, I could get most of the work done (and much faster) by using outboundBasicLink or inboundBasicLink to pull together lists of links by type. (e.g. Inbound links of type “directed” always originate with the director of the film.) 

After a quick two hours of work, I had a set of sensible HTML “notecards” that interlink like a wiki. I uploaded them to a folder on the server and voilà. A set of notes on the web.

Export is daunting at first, but once you figure out the basics (or at least, the basics that you need), it’s incredibly powerful and insanely fast. And so I walk away feeling like, with a bit of creativity and some careful thinking, I could make it do anything…

Process is Part of the Project

Québec’s response to the pandemic seems to be working. This is great news, but it’s also a sign that over the next few weeks there will likely be more of the quarantines and shutdowns that are keeping the pace of new infection so low. With this in mind, yesterday I decided to try to get some work done.

The dull anxiety of the past week — low-grade and barely noticeable, but there like a spare battery (remember those?) weighing down your backpack — had deadened my mind too much for serious reading. So I opened up the TBX file I’ve started for my new book project and decided to piddle. 

First off, I knew I wanted some notes that will serve as the starting point for some mini-analysis on early materials. I created them — simple titles with no text— and slipped them into my “To Be Filed” container. I’ll get back to them when my mind’s working better. 

From there, working in outline view:

  • I decided to organize some of the chaos in my initial bibliography. There are a couple topics I need to know more about as context. Book and article notes about them currently clutter my sources container — I can’t see anything else — but I know that they are quickly going to get lost as that container fills up with other things. So I created a note for each of these topics and went through my sources container making links to the notes for each of the sources that speak to those subjects. So I now have single note “index” pointing me toward sources as a starting points for when I’m ready to use them.
  • For the index note on the topic for which I have a lot of reading to do, I organized the list of links into sensible groups and then sequenced the groups in a sensible order. Now I have a reading plan ready when I eventually return to this.
  • As I was organizing the bibliographies, I went back to my mini-analysis note on Miramax and dropped in a couple links for important sources I’d remembered when I ran across them as I was working. I also decided to add a few sentences to the note explaining to myself what my interest in the company was (and wasn’t). Because I risk becoming seriously distracted and want at least some marker for later of what I initially thought it was I was doing. Then I jotted down some basic technical information I had at hand so I’d have it later without looking for it. It probably goes in its own note, but I’ll sort that out later.
  • I looked at some links between films, realized they were pointed in the “wrong way” — not that I’d made a mistake when I made them but that when I made them I’d been thinking badly about what they’d show — so I swapped them around. As I was doing this, I noticed some comments jotted in a few notes about allusions between some films and novels. Sorting out the direction of the mistaken links helped me see how I could formally link the notes for these alluding texts without “just linking everything.” So I made those links.
  • Curious, I selected one of these notes and switched to Hyperbolic view. As I poked around what is still a very limited link network, I noticed that some notes I expected to see there were missing. Switching back to the outline, I added links tying these notes into the network and then switched back to hyperbolic view. Now things were a mess — everything was connecting with everything — but I wasn’t sure how to fix it (or if it’s actually broken). So I left things as-is and went back to my outline.

Suddenly, my phone dings. My brother’s texting. I look at the clock and have been at this for an hour and a half. I scroll through the outline I’m working on. There’s still lots to do and I’m not sure how to go about it. However, things are better than when I opened the file in the sense that I’ve made some headway on clarifying my initial research questions and plans.

So I call it a day.

Outlines and the Terrible Beauty of Maps

After years of relying on map view in my TBX files, I’ve gotten to the point where I generally use outline as my default. The problem I have with maps is that they cue an aesthetic response that overrides other concerns, and I have trouble setting that response aside. A map is either beautiful and this creates a barrier to revision, or it is ugly and making it attractive becomes my priority. If I leave it ugly, then I find it hard to work in the file unless I stop using the map. Outlines short circuits this enormous weak spot in my mental make-up.

So when do I use maps now? Usually only when a file or project has developed to the point where I know what it is and how it’s working. I then create maps that operate either like a publication of key aspects or like an interface for interacting with attributes that change over time. In both cases, attention to aesthetics becomes an asset rather than a distraction because a beautiful map will likely be legible while being dense with information. Colors, shapes, borders, badges, even shadows can be used to communicate content at a glance, and in this context, attention to aesthetics makes them communicate clearly.

What this means in practice is that generally maps are for “reading” my materials, while outline and attribute browser are my work views. And hyperbolic view? I’m intrigued by it and flipping to it more and more, but I haven’t quite wrapped my head around what it does for me yet.

Crosspost: And then there were permalinks

I thought I’d wait to deal with permalinks, but I had an idea while taking a walk and decided to give it a try. Five minutes later they’re done. I always forget how crazy simple HTML export is. The key is to start with nothing and only build the things you understand. Then go from there.

I’m also not messing with an external CSS stylesheet. I just have a CSS note in my TBX file and a line in the page-head element of the templates that pulls that note’s text in as internal CSS for each page on the site. So I have all the advantages of external CSS when I’m working locally, but I don’t have to keep track of a stylesheet on the server. It’s the best of both worlds.

Crosspost: An Earlier Blog Experiment

I tried to run a blog from Tinderbox once before, but in that case, I tried to duplicate the look and functionality of my fairly complicated WordPress set-up. This meant building a TBX file that was a big hairy monster of a machine. I’m proud that I got the thing working, but running it took too much energy, and I wisely retreated to the CMS version of the blog after only a few days, lesson learned.

A New Wiki Project

In 2013, I experimented with using a wiki in some of my classes for the first time. In those first experiments, I was learning what a wiki could be used for, how students interacted with them, how to fit them into the other assigned work and, most alarmingly, how to manage and host a site used by dozens of people simultaneously. It was a lot and things changed quickly as I learned and improved.

Since those first projects, I’ve never not included a wiki component in at least one of my courses each semester. My expectations have changed dramatically though because students responded without the enthusiasm that I naively and laughably assumed they’d have. (Hope springs eternal, right?) In fact, students were often openly resistant to the project for a variety of reasons. Some of these were:

  1. a sense that the wiki made a course they took to fill a requirement more difficult than it would have been if they’d taken something else.
  2. a genuine ignorance about how to use a computer for anything other than opening a browser and clicking on links or opening a word processor, typing with minimal formatting, and then printing.
  3. a distaste for the aesthetics and UI of a site that was different from Facebook, Instagram or [fill in the blank].
  4. a sense that the old guy in the front of the room was trying to “play computer” with the kids and didn’t get that there was an app that did [whatever the day’s assignment was] so much better than the wiki did. “Maybe we should be using that, sir?”

A lot of my energy in the early years of using a wiki was spent figuring out how to get students past these initial objections to the project.

Early on, I made the first of these worse by being unable to explain what the wiki was for and how it would help make the course better in terms students could understand. I knew why we were doing the wiki and what it was for, but what I knew was embedded in and dependent upon a context my students didn’t share. So I was still doing the basic pedagogical work of figuring out how to speak what they needed to know in terms they could understand. In terms of assignments and requirements, I was working from hunches, experimenting and it would take me some time to get a handle on both what to assign and how to explain why I was assigning it. After the first two wiki projects, I scaled back expectations so I could figure out through experiments how to speak clearly about what we were doing and why. This took some time, but I think I’m there now.

The second problem, student’s inability to use a computer—which was often (nearly always) paired with an obsessive, continual use of social media on their telephone—caught me off guard, but it was easier to address than the first. A wiki is perfect for posting how-to instructions and examples. I wrote and recycled these year after year and now have a set of mark-up pages ready to drop into each new wiki. These work.

The final two objections I kept running into were different in kind from the first two. In both three and four, students, confronted with something they didn’t want to do, were offering a reason why the thing they didn’t like was in fact crippled by a failing that made it worth ignoring. The first semester I ran the project in a class, I took these objections about aesthetics and alternatives at face value without recognizing that this was just a familiar classroom swap: the real student-side problem—I don’t want to do this—was being replaced by a teacher-side problem with the assignment. This isn’t mean-spirited or manipulative on the students’ part. It’s just being a student and is exactly equivalent to complaints that a book is boring. I don’t change the booklist to match what they want to read; instead I explain why it’s worth reading what we are. But it took me a few semesters to realize that, confronted with complaints about the wiki being ugly or awkward or old-fashioned, I just needed to explain why we use a wiki rather than [fill in the blank] and what the look and UI of the wiki make possible. This responds to the real problem—I don’t want to do this—by showing them why doing the project is useful. This too works.

I’ve kept using the wiki all this time because I think that exposing students to structured writing and to hypertext is valuable and that the combination of metacognition and practical skills required by the project equip them to be better students. Now, has that been the outcome semester after semester? No. But I’ve never had a semester where the wiki has been a failure. Students figure things out and do the work and usually do it well, which given its scale and how difficult I found it to organize and manage initially feels like a success.

(Don’t get me wrong. Sometimes things have been incredible. One year for example, through a quirk of scheduling, I taught an intact group of students two semesters in a row. We built a wiki together in the Fall and they reacted with familiar dismay and crocodile tears over having to do extra online work in an old-fashioned platform. Everything worked, but they were a vocal group and, when I was planning the winter term, I decided not to include a wiki on the syllabus. But then, three or four weeks into that second semester, they asked if there was anyway to reactivate the previous semester’s wiki because they were annoyed at having to organize and prepare the work they were now having to do without the wiki because doing the work was harder without it. Now I don’t want to exaggerate: what they wanted was mostly the ability to coordinate group work with shared buckets of materials and lists that they could edit together, things in other words that they could do with something like Google Docs. Still, they were thinking of these tasks in terms of a persistent set of linked documents rather than one-off document lists. This was important and I was happy.)

So looking back, what I see in these experiments is a productive encounter between my initial enthusiasm for what I believed possible with the project and the predictable, not completely unreasonable questions from students about why doing work on the wiki was worth their time. I’ve learned to address these and in the process, I think I’ve learned better how to use the wiki in class, how to integrate it into required classwork, and also how to present it to students.

Now a new semester is starting, and using these experiences, I’ve completely rethought the wiki project. The new project is more extensive and is more tightly integrated into the core assignments of the courses, both of which are big changes. I’m writing about it here now because what I hope to do over the course of the semester is to speak about those changes in some posts and comment on how things have gone.

But as a starting point, I wanted to look back over the past few years, partly as a transition, yes, but partly as an acknowledgement of the completely unglamorous work of sticking with the wiki long enough to get a sense of how to use it better. And perhaps most importantly, it’s a way to acknowledge that work depended upon the cooperation and support of students who sat in my course doing their best to make things work as I figured things out.

So acknowledgements made, it’s time to move on and to talk about how the new project is put together and how it works out. I’ll get to that soon…

Tinderbox RSS

This blog isn’t a tech blog, but I write about Eastgate’s Tinderbox semi-regularly.

To help those of you who are interested in the TBX posts but not necessarily all the rest, there’s now a link to the TBX-only RSS in the sidebar. (Or click here.)

ps—I suppose this is fair warning that the next few posts involve me over-investing in not-fancy TV and movies.

Exporting Text from Tinderbox

At the same time as I was finishing up work on the new version of my site using Tinderbox, a few threads popped up on the Tinderbox forum talking about the perceived difficulty of Tinderbox export. This got me thinking yet again about the source of all the trouble. At various times and in various moods, I have both agreed export could be a pain and been astonished at how easy it made creating complex documents. Thinking about it again led me to jot down metaphorical thoughts about car dealers and Mad Libs. I also tried to describe what I see as the fundamental difference between TBX’s export and other common tools.

Once I had those ramblings out of my system, I decided to use what I’d learned making my web site file to offer up a short series of posts that tries to show how I’ve come to think about basic export. Obviously there are other examples, instructions and information about Tinderbox available elsewhere. (The TBX help files, Mark Bernstein’s The Tinderbox Way, Mark Anderson’s TbRef,  Welcome to Sherwood, and the videos of Dominique Renaud are only a few). I try not to repeat that work here.

a map of the series

Instead, what I’ll try to do is show how working backwards from the desired output rather than forward from a note is a useful (and manageable) way to think about export. In my opinion, working this way resolves a lot of the difficulty I initially experienced.

Each of the examples in these series will begin by creating a concrete instance of the desired output. Once this instance is written, I show how to transform it into an export template that will generate the same output from any note. I’ll be using the same sample file throughout (download .zip).

You should start with exporting a form letter. All the other examples take the information it provides for granted.

You can find the rest of the examples here:

In the next few days, I’ll close out the series with a few thoughts about what’s involved in exporting to the web.

Challenges Exporting in Tinderbox

A thread on the Tinderbox backstage forum has touched on export and its difficulties. I’ve written here that export is surprisingly easy, and I still think, in practice, that it is. Yet there are real conceptual difficulties that can also make it feel mysterious and intimidating.

I’ve been thinking about export a bit these past weeks because I was so dissatisfied with my explanation of how my grading rubric worked. Rereading what I’d written, I saw that I had never managed to figure out a way to explain how I thought about export. So what I wrote wound up assuming that the person reading already knew much of what I hoped to explain. I’ve been trying figure out what I would have needed to say in the weeks since.

What follows is my tentative list of what I see as my conceptual challenges as I started dealing with export. They are not shortcomings in the export process itself.

Experience

I mentioned this difficulty in the backstage thread, and it’s pretty straightforward: I know how to organize a word document because the arrangement of info on my screen is synonymous with its arrangement on the printed page. My output is organized by my workspace even in programs as flexible as Scrivener: ultimately, the sequence I impose in my binder determines the arrangement of my output document. If I want a different output, I rearrange my workspace.

TBX notes and output don’t march in lock step in the same way. They can (and do in the built-in templates) but the in-doc organization of notes and attributes will often be largely independent of export results. That makes TBX stand out: I can’t think of another program on my computer that divorces workspace from output organization in this way. The separation is powerful but uncommon enough that, when I start a new export, I lack personal examples or comparable experience to draw on and feel adrift (at precisely the moment—“Print!”—when I usually feel done).

Direction

I should say right off the bat, that this problem may actually be simply a misunderstanding resulting from my lack of comp sci training. It’s persistent enough that it feels real though. I’ll explain with an analogy to a stamp action.

Assume I have a note with two Attributes $A and $B with respective values of 1 and 2. I make a stamp with the action $A=$B and apply that stamp to the note. The two Attribute values will now be the same. But will the values both be 1 or will they both be 2? The action’s behaviour is stable and unchanging. So there is a correct answer. (I think it’s 2?) But my point is this: the direction of the equals sign is neither implicit nor obvious. It’s arbitrary, and ultimately, you just have to learn it. Which is fine because I have no natural sense of the direction of the equals sign.

I do, however, have a sense of the natural direction of export. In my experience and in the common understanding of the word, export moves from me (and my data) and toward something (or someone) else. I send a document to my printer, not the other way around. As a picture, my daily experience of printing is:

[ME with my organized info]    ::push:: ==>              [printer]

TBX export feels to me like it works in the opposite direction. I start the process by creating a template, the symbolic image of the output I want. This image is built up by pulling information piece by piece from my notes and then rearranging it for export. As a result, TBX feels like:

[my organized info]       ==>::pull::     [ME organizing a template of desired output]

The shift in direction from pushing to a printer to pulling from notes seems like nothing, but I think it’s at the root of a lot of my uneasiness when I start a new export. Things are flipped over onto their head.*

Templates

Tinderbox hides a very unusual, very peculiar thing under the familiar and innocuous word, “template.”

In most common applications, templates are where you turn for an easy solution. Need some colors that match in a document? Pick a template. Have to write a report and don’t know how to format it? Pick a template. And when you go to choose the templates, you are presented with a preview image. WYSIWYG!

TBX templates are nothing like this. Most obviously, they aren’t pre-fab except in very basic forms included for import. Instead, you make the template you need in order to produce the output you want. This is great and powerful (Oz-level great and powerful even), but different from the common experience.

Less obviously and less trivially, templates:

  • are usually a system of notes (rather than a single object) structured hierarchically using path names;
  • produce output as a collage of your data using instructions that are distributed across the system of notes:  your output is a product of interactions across the system as a whole;
  • represent the desired output symbolically rather than mimetically. This shift from image to symbol is extremely powerful but it makes greater cognitive demands on the person confronted with the symbol. It can be challenging, for example, to understand the connection between the distributed instructions I’ve placed in a template and the output preview visible on the “preview” tab. (I teach literature and see this difficulty “reading through” symbols play out every day in class.)

Taken together, I think all of this highlights that an uneasiness with export may express a pedagogical need rather than a technical failing.

Translating Computer Science

I wouldn’t want any of this to seem to suggest there is anything wrong with TBX export. I don’t believe what I’m describing here arises from technical or interface difficulties (or at least none I could imagine or suggest a solution to). They seem to me to be conceptual.

TBX takes what this layman considers to be high-order computer science concepts and translates them into a tool I can use to great benefit. But translation will only get me so far. Ultimately, I have to wrap my head around how the tool works, and the information I need to do so (HTML, CSS, page-layout concepts, etc.) may be out of scope for a TBX tutorial.

Export is powerful tool and, when I’m done it with, feels like it was technically easy to manage. But an important aspect of that power is completely (and refreshingly) unfamiliar: TBX asks me what I want it to produce and stands ready to make anything and everything that I have so far asked it for. But answering that initial question—what do I want?—positively and for myself (rather than simply selecting from a range of choices) is disorienting because I’m so seldom asked it elsewhere. And so I’m often still wrapping my head around what my answer might be when I sit down to make a template.


*–rereading this it occurs to me that it might be useful to think of TBX as operating a bit like an old newspaper. There’s my workspace which is the newsroom with all the action that entails. But then off to the side, out back maybe, there is layout and the printshop, which together compile, label and produce the day’s paper.

In terms of the analogy, I never actually “export” anything. I’m really just sending the work I’ve got at a particular moment over to the printshop for inclusion in the most recent edition. And that edition will, obviously, be organized according to constraints and goals (page count, ad space, etc.) that have little to do with how work is organized in the newsroom.

Exporting Comment Sheets

This is my final post about my grading rubric, and I’m going to use it to explain how I set up my export template to create comment sheets. But before I do that, I’m going to take a moment and explain why I think export can seem difficult. If you’re not interested, you skip it.

Export Expectations

I’ve commented elsewhere about discovering how simple and flexible Tinderbox export is. I still believe this is true. Yet, every time I sit down to export something new, I find the task daunting. What gives?

I’m convinced the problem is conceptual and arises from the uniformity of the export function in every other application I use. In those programs, export is called “Print” and it functions in a single, predictable way: from top left to bottom right, line-by-line. Most importantly, this export generates a copy of the arrangement of materials visible on my screen. The trade off for this simplicity is that, because document organization and export organization are synonymous, I must organize my document according to the fixed rules of printing. (This is a constraint we all live productively with most of the time.)

Tinderbox does not work in this way. I can organize my notes however I want and from that organization create whatever export file I want. Working in a file and exporting from that file are largely independent process. In a lot of ways that’s a very good thing. Something like my course map wouldn’t be possible if I were constrained by the rules governing “Print.” The cost of the flexibility can, however, feel considerable when the few default export options aren’t enough and I’m confronted with the problem of how to imagine and to create a “Print” that is adequate to my needs.

Oddly enough, I don’t think that the difficulty involved in coming up with “Print” is found in the export code. In my opinion, export code is much simpler than agent queries and action code. So what’s the problem? Crazy as it sounds, I think it’s having to decide what I want to have once my export is complete. I don’t have to do that in MSWord because — like Henry Ford and his Model T — Word gives me one option: a series of 8” x 11” pages printed top to bottom that match what I see on my screen. TBX is no Model T: the available options are seemingly, overwhelmingly limitless and choosing is hard.

Getting Started

My trick for getting past this block is to step away from my computer, to grab a pencil and paper, and to sketch out what I want. Once I know that, I’ve found that making TBX give it to me is pretty easy. And if I change my mind and want to tweak something? That’s usually easy too.

For the comment sheets I was exporting from my rubric, I wanted various note texts and attribute values to be collected and presented in a sensible way. I thought about this in terms of the rubric hierarchy and came up with a structure shown here.

 

What I Need
General Structure

In this image, the yellow box would be summary information, all of which I had stored in the root note of each instance of the rubric. The green boxes correspond to information stored in various attributes of the criteria notes; the blue boxes are my comments, stored as note text in the descriptor I checked while grading. To create this output I’ll use three templates all of which will be based on the basic HTML template provided through the FILE menu.

Before getting started with the templates, I should also mention that my pencil drawing of the comment sheet indicates where in my TBX notes the information I want to export is located. To give a sense of what that means, I’ve taken screenshot of my sample comment sheet and used colored boxes and labels to do the same thing here.

Annotated Comment Sheet

Templates

I import the html templates that comes with TBX and use them to create my starting templates by copy/pasting them as I need to. By the end of this process, each note in the rubric needs to have the appropriate template assigned to it, either directly or by the export code. To assign them directly, you can use either the HTML tab of the inspector (cmd-1 then the tab marked”< >”) or the HTML selector tab above the key attributes in the text pane. (If the selector tabs aren’t visible, you can turn them on with “WINDOW–>SHOW TEXT PANE SELECTOR.”)

t_comment_sheet

My first template is assigned to the root note. It collects and arranges information from this using three basic export codes. These are:

^title^
^value($RubricEssayGrade)^
^text^

^title^ and ^text^ are easy: the first prints the $Name of the note (which my rubric prototypes have already set to the $ExportFileName) and the second prints anything I’ve written in the $Text attribute (which is what is displayed in the text pane). These two codes are special: they pull the information in $Name and $Text without requiring the attributes to be named. For all other attributes, you use ^value()^, placing the attribute name in the parenthesis. In my template, the ^value()^ code will insert the student’s grade for the essay.

If these three terms were simply listed in my template, then the export would be an unformatted list with no context and would look sloppy. To fix this, I add some stock text — a “/100” after the grade and a generic introduction to the criteria comments — and format everything using html tags. My understanding is that most (all?) html tags will be recognized during export, but I only know a few. So I can’t say for sure. Text added to the template will be added to the output as typed. No codes necessary.

Finally, I want the export to include more than the root note of my rubric. So I need to indicate that my criteria notes should be exported as well. These criteria are the children of the root, so I can use the command:

^children^

That one command is enough to tell Tinderbox to include each of the children in the export. And what information about the children will it include? Whatever the child’s export template says should be included. In my template, I assign a template for the children’s export in the export code by adding a parenthesis to the ^children^ command. But if you’ve assigned the correct template to the criteria using the inspector or the HTML selector tab that’s not necessary.

This template will generate the text found in the red, the blue and the first two green boxes in my image above and, assembled, looks like this:

Comment Sheet Template
Comment Sheet Template

t_comment_sheet_category

This template controls the export of the criteria notes. It uses the same export codes as the previous template, and uses similar html tags. The stock text is different and integrates an attribute value in a more interesting way, the path for the template used to treat the criteria’s children has also been updated, but otherwise, it works in exactly the same way.

This template will generate the text found in the pink, the orange, the brown and the last green boxes, and, assembled, looks like this:.

Criteria template
Criteria template

t_comment_sheet_text_item

The final template, which controls the export of descriptor comments is a good example of the daunting/easy paradox of export. On the one hand this template needs to do only one thing: export the descriptor’s note text. To do this requires nothing more than the single export code ^text^. Easy-peasy. (I spice things up by adding the stock text “Your essay: ” before the comments text because I want things to look classy.)

But there’s a hitch: I only want to export the text of the descriptor note I checked when grading. ^text^ on its own will include all the descriptor comments whether I checked them or not. I don’t know how to do that. So it’s off to the TbRef and the user forums because there’s no way to intuit the answer. I know I need an export code that does the same thing as the action code I used when doing calculations, and I have to look or ask until I find it. It doesn’t take long, but if feels harder — and is more daunting — than selecting “Print.” Once I have the command I need — ^if($Checked==true)^ — I add it and everything works.

The resulting template generates the olive box above, and although it has felt like the hardest to make, it is only two lines long and pulls information from only one attribute. So it is actually the simplest and the shortest of the three.

Descriptor Template
Descriptor Template

 

Final Step

In practice, when I’m done grading, I move all the rubrics from out of the student containers and into a new container that, when I export, will be a folder with all of the correctly named comment sheets inside. I batch convert these to pdfs and upload them to my college’s course management system and students can consult them from there.

The one hitch I see in this output is that the default text sizes for the text is large when printed, but I haven’t decided if I want to do anything about that. If I decide I do, I think that all that’s required is a simple html command in the root template. But that said, I’m pretty sure my students reading their comments on a screen rather than on paper which means there’s probably no problem.

Rubric: Calculating Grades

Grades are calculated automatically in my grading rubric using attribute values assigned or calculated at each of the three levels of the hierarchy. So to explain how it works, I’m going to start with the lowest level and move up to the highest.

Before I do though, it’s perhaps worth noting that these calculations use the $Checked attribute. If checkboxes aren’t visible in the outline, you can show them by going to VIEW ==> USE CHECKBOXES. Or if you prefer, $Checked can be set as a key attribute in the p_rubric_component prototype.

Descriptors

Basic grade values are established here by selecting one of the available descriptors and ticking its checkbox. If you wish to change the default point value of a descriptor, you may do so by entering the new value in the key attributes. The rule presented in the previous post will update its $Name to reflect the new value.

Two things are worth noting:

  1. Once the grading is done, there should only be one descriptor checked per criterion. The final step in the grade calculation is a simple sum: if more than one descriptor is checked, the student will be awarded points more than once for that criterion, and their final grade will be higher than it should be.
  2. The criteria and final grade calculations that I use assume that each descriptor is evaluated by a point score between 0 and 10. If this isn’t the case for you, you’d have to adjust the calculations.

Criteria

The criteria-note calculations take the point value of a checked descriptor and multiply it by its criterion’s weight to determine the points awarded toward the final grade. (See note on weighting below.)

To generate my weighted grade, I need to multiply the assigned grade — stored in the selected descriptor’s $RubricPointValue — by the criterion’s weight — stored in as the criterion’s $RubricCategoryMax value. The simplified calculation that would seem to perform this multiplication (but doesn’t) looks like this:

$RubricCategoryPointValue=$RubricPointValue(child)*$RubricCategoryMax

This tells TBX to pull the point value stored in the child note (i.e. the descriptor), to multiply it by the criterion weight stored in the criteria note, and to store the result in $RubricCategoryPointValue.

However, there is a problem: each criteria has more than one child. This rule doesn’t care about that and simply calculates the weighted value of the first child, whether I’ve checked that child or not. To make my rubric work, I have to tell TBX that it should use the $RubricPointValue of the child I have checked and to ignore the rest.

Using Mark Anderson’s TbRef, I discovered that the way to do this is to use “sum_if(x, y, z)” which I found very intimidating at first but which I’ve decided is not as complicated as it seems. As I understand it, this command tells TBX to add up numerical values that match a set of conditions (or arguments) laid out in the parenthesis. There are three types of conditions possible, and I’ve labelled them x, y, and z. (I don’t know if all three are required but I use all three of them in my calculation.) What they stand for is this:

  • X tells TBX which notes to look at by indicating a location. For example, I could replace X with “all” and TBX would look at all the notes in my project, but for my rubric, I only need my calculation to consider “children.”
  • Z tells TBX which attribute to look at. In my calculation, I want TBX to see if a descriptor note is checked or not, so I use “$Checked” for the value of z.
  • Y tells TBX what value it’s looking for in the attribute it’s examining. Now I’m sure this argument allows the initiated to do incredible, magical things, but thankfully I only have to ask TBX to look for checkmarks, which is a boolean value — yes/no, true/false — and so, I need nothing more complicated than “$Checked==true”.

Combine these three conditions in a parenthesis, separate them by commas and then use them to update the equation presented above and you have:

$RubricCategoryPointValue=sum_if(children,$Checked==true,$RubricPointValue)*$RubricCategoryMax*0.1

This looks complicated enough to make my heart flutter and my stomach ache, but it works and in a weird sort of way is logical. You just have to work up step-by-step. (…and ask questions.)

Regarding the final term of the equation (“*0.1”): because I use values from 0 to 100 to define criteria weighting, I have to multiply the product of my calculation by 0.1. If I don’t do this, the final grade calculation will report a grade of 88/100 as 880/100. Defining criteria weight with a scale of 0-10 would eliminate this problem, but I prefer thinking of weighting as percentages, which is more natural with a 0-100 scale. Multiplying by 0.1 moves the decimal point to where it belongs.

The Criteria-level Rule
The Criteria-level Rule

This calculation is stored as a rule in the p_rubric_category prototype, which then automatically applies it to all criteria notes in all instances of the rubric.

Final Grade

In my rubric, all the heavy lifting happens at the criteria level. Once the weighted point values are calculated and stored in the $RubricCategoryPointValue attribute, all that’s left to do is to add up these weighted values. To do this I use the “sum(x,z)” command. In this command, X and Z function in the same way as they do above. No Y argument is necessary because the command isn’t evaluating a condition in order to determine which values to add: “sum(x,z)” adds all the values found at the designated location.

As a result, the rule that calculates the final grade is simpler and reads:

$RubricEssayGrade=sum(children,$RubricCategoryPointValue);

The result of this sum — which will always be a value between 0 and 100 — is the final grade of the essay stated as a percentage. It is stored in the $RubricEssayGrade attribute where it is used to set the rubric’s $Subtitle and $HoverExpression.

The Root-level Rule
The Root-level Rule

And that’s it for the grade calculations. In the next post, I’ll explain how I export the rubric as a comment sheet for students.

****

Note on weighting

If all the criteria were weighted the same or if the descriptors awarded points on scales that directly reflected differences in weighting, the criteria-based calculation would not be necessary. The root-level rule could use sum_if(x,y,z) to calculate the final grade directly. However, I choose to use a standard scale (0 to 10) to evaluate all criteria and then to assign the weight for the criteria in a separate attribute.

I prefer using multiple attributes for two reasons. First, the standard scale makes grading easier: I know “7” always means 7/10 (and not 7/15 or 7/20). Second and more importantly, having a separate attribute for the weighting allows me to adjust (and experiment with) the relative weighting of my criteria without out having to continually adjust the individual point values of dozens of descriptors. This makes the rubric much easier to adapt for use in new assignments.

 

Rubric: Prototypes & Agents

In this second post I’m going to go through the prototypes and agents I used to make my grading rubric. When I’m done, I’ll have explained everything that doesn’t have to do with grade calculations or export. (Subsequent posts will take up those topics.) The result is a long post that is predictably schematic. If you’re reading, I’ve assumed you’re interested in nuts-n-bolts.

Before I get started though, I should make clear that, in what follows, when I say “the rubric” I’m referring to a prototype that contains a two level hierarchy. Because my rubric is a prototype (called p_essay_rubric) it sits in the container where I store the rest of my prototypes and appears as an option in the assign prototype contextual menu.

The rubric is a prototype
The rubric is a prototype

When I assign this prototype to a new note, that note automatically duplicates the entire prototype hierarchy, providing an editable copy of the rubric that I can use to evaluate an individual student’s work. I will call these editable copies “instances of the rubric.”

I should also mention that the rubric relies upon seven user attributes, each of which is assigned to only one level of the rubric hierarchy as a key attribute.

User Attributes
The user attributes

Of these user attributes, three of them–$RubricCategoryPointValue, $RubricCategoryMax, and $RubricPointValue–are number values used to calculate grades. The rest hold either sets or strings.

The Rubric Hierarchy

The rubric is organized with three levels of hierarchy.

The Root (“The Rubric”)

The top level of the rubric hierarchy has three key attributes: $RubricEssayGrade, $RubricAssignmentName, and $RubricExportFileName. All of these are set automatically by rules, agents or on-add actions. The $HoverExpression is set to display $Subtitle. (A separate rule sets the $Subtitle to the final grade. See below.) The note’s $Text will be presented as a general introductory comment in bold italics on the exported comment sheet. It is blank in the rubric but can be edited in each individual instance of the rubric. If no comment is entered, the export will insert a stock intro. This top-level note is a prototype and is named p_essay_rubric.

Criteria

The second-level notes identify the criteria used to evaluate the essay. There is one note per criterion.

The criteria in my rubric
The criteria in my rubric

Each note’s $Text contains a brief description of the criterion. When the final comment sheet is exported, this text will be presented as an italicized introduction to the specific feedback. $RubricCategoryPointValue and $RubricCategoryMax appear as key attributes. $RubricCategoryPointValue is calculated by a rule, but $RubricCategoryMax is set manually and establishes the relative weight of the criteria in the calculation of the final grade. (A value of 5 assigns the criteria grade a weight of 5% in the final grade.) Criteria are assigned the prototype p_rubric_category.

Making Revision to the Criteria: Evaluation criteria will likely change when the assignment or the course changes, and so both the number of criteria and their descriptions are fully editable. However, as I have set things up, the grade calculations assume that the sum of the $RubricCategoryMax values across all the criteria will equal 100. To ensure calculations are correct, any changes to the number of criteria or to the max value of individual criteria should be accounted for in changes to other notes’ $RubricCategoryMax attribute. Although $Text descriptions of the criteria can be edited freely, once I have begun grading I do not touch them so that each student’s instance of the rubric will match all the others.

Descriptors

The bottom level of the rubric hierarchy contains descriptors attached to point values. You grade an essay by selecting one descriptor within each criteria by ticking it’s checkbox. $RubricPointValue and $RubricCompDescriptiveName are set as key attributes. Although each note has a default value for both attributes set by the rubric prototype, they may be edited freely. The $Name of each of these notes is set automatically as a composite of the key attribute values and will be updated to reflect any changes made. The note $Text will be exported as the principal feedback on the comment sheet, and so, descriptor texts are completely flexible.

Sample Descriptors
Sample Descriptors

It is possible to have as many or as few descriptor notes as you like for each criteria. There can even be different descriptors that are assigned the same point value. Descriptor notes are assigned the prototype p_rubric_component.

Prototypes

The rubric hierarchy relies upon four prototype notes:

  • p_student
  • p_essay_rubric
  • p_rubric_category
  • p_rubric_component.

Together these prototypes:

  1. recreate the prototype’s hierarchy of notes for each instance of the rubric;
  2. set key attributes in each instance of the rubric;
  3. set a variety of rules and on-add actions that are used to define consistent attribute values in instances of the rubric;
  4. distinguish between levels of the hierarchy, allowing agents to act upon specific sets of notes;
  5. set rules that calculate grades based on checkbox selections in an instance of the rubric.

What follows is a brief explanation of each of the four prototypes.

p_student

The p_student prototype is not a part of the rubric itself, but I always create instances of the rubric as a child of a student note. (In my course file, I keep one note per student as a record of ID number, email address, etc.) It makes sense to use on-add actions set by the student note prototype to fill in basic attribute values for the rubric.

The first on-add action links the instance of the rubric to a specific student’s work by having the rubric pull its value for $s_LastName from the student note:

$s_LastName=$s_LastName(parent)

By setting this value immediately, I avoid the situation where I forget to label a rubric and can’t figure out to whom the comments should go.

The second action creates an instance of the rubric by setting the prototype of the note I create to p_essay_rubric:

$Prototype="p_essay_rubric";

Finally, I set $RubricAssignmentName:

$RubricAssignmentName="Essay One (Moby Dick)";

This last action has to be updated each time the rubric is used for a new assignment. It could probably be automated somehow, but I haven’t taken the time to figure out how. I don’t mind changing the string manually when I move from one assignment to another. Together, these actions look like this on the action tab of the p_student inspector:

On-add actions for student note
On-add actions for student note

p_essay_rubric

I began this post by pointing out that my rubric is a prototype. This is that prototype, and except for its $Name, it and its descendent are identical to the instance of the rubric I use in the screencapped demonstration included in my last post.

The prototype hierarchy
The prototype hierarchy

This is possible because the attribute $PrototypeBequeathsChildren is set to true for this prototype. Setting this attribute to true means that when a new note is assigned this prototype, the criteria notes inside this prototype—along with their text, their rules, their on-add actions, etc.—will automatically be created inside the new note as well. Because student notes automatically assign this prototype to their children, all I have to do is create a new note as a child of a student note to create complete instance of the rubric.

[Update: if you are thinking about using prototypes to bequeath children in this way, it’s very important that none of those children are prototypes in their own right. Instead, assign the children prototypes that are outside the bequeathed hierarchy. The TbRef entry on prototypes can give additional info.]

This attribute allows prototypes to create notes in a container
Creating notes in a container with a prototype

In addition to setting key attributes, this prototype contains one non-calculation rule:

$Subtitle=$RubricEssayGrade+"/100"

This sets the subtitle to show the final grade as a fraction over “/100.” It could just as easily be set to display it as a percentage labelled “%”. I’ve also set the hover expression of this note to $Subtitle.

I’ll explain the calculation rule in the next post.

p_rubric_category & p_rubric_component

These prototypes are assigned to criteria and descriptor notes respectively and are relatively simple. They set key attributes and identify the notes in a way that can be queried by agents. In addition, p_rubric_category assigns a calculation rule. As with p_essay_rubric, $PrototypeBequeathsChildren is set as true for p_rubric_category. That’s it.

Agents

The final piece of the basic layout is a set of three agents.

a_set_rubric_ExportFileName

The most important of these agents sets the file name that will be used when the comment sheet is exported. The agent queries for the p_essay_rubric prototype and then performs this action:

$RubricExportFileName=$s_IDNumber(parent(original))+"--"+$s_FirstName(parent(original))+"-"+$RubricAssignmentName

In the demonstration I included in my previous post, this action produces the file name: 123456–Brian-Essay One (Moby Dick).

Here’s how.

First off, everything looks simpler if we ignore the parentheses for a moment. If we do, it’s clear that the opening number sequence is being reported as the value of $s_IDNumber and “Brian” is being reported as the value of $s_FirstName. I’ve formatted these two results to make them easier to read by separating them with an m-dash by adding:

+"--"+

I’ve used the same technique to place a dash between my name and the assignment name later on.

Now those parentheses…

I only figured out what to do here with help offered on the TBX user forums (link to thread). Using my agent’s action, I’m going explain what I learned there. To make things clearer though, I’m initially going to ignore everything from my agent except the ID Number.

Consider the following action:

$RubricExportFileName=$s_IDNumber

This action will cause the export file name assigned to a note to be the same as the ID number assigned to that same note. I can’t use this action in the rubric because the student ID number is stored in a student note while the export file name is stored in the rubric. To make the rubric’s export file name use the container’s student ID number, I need to point the action to the information I want in the student note. So I tried doing this by pointing the action toward its parent:

$RubricExportFileName=$s_IDNumber(parent)

Unfortunately, this doesn’t work either. Why? Well, agents work on the aliases of notes that they gather using their query (and not directly on the original note). So when my agent action pulled the ID Number from the parent note, it wasn’t pulling the information from the parent of my rubric. It was pulling it from the parent of the alias created by the agent, which is the agent. And the agent didn’t have any student ID information. What I needed was to point my action not to the parent of the alias but to the parent of the original note. To do this, I need this:

$RubricExportFileName=$s_IDNumber(parent(original))

Make the same changes to the first name attribute, and I end up with an agent that set the needed export file name by pulling information from three different attributes.

Action to set export file name
Action to set export file name

a_set Rubric Component Name & a_set_rubric_Name

The remaining two agents simply maintain note names. The first queries for the prototype “p_essay_rubric” and sets $Name using the export file name:

$Name=$RubricExportFileName;

The second updates the $Name of a descriptor to reflect any revisions to the point value or the descriptive name. It queries for the prototype p_rubric_component and then performs this action:

$Name=$RubricPointValue+”: “+$RubricCompDescriptiveName;

Because the attribute being set by this action is part of the same note as the attributes holding the information it is using, there is no need for parentheses this time.

Wrapping Up

So this has been a long post, and a boring one too I think, but hopefully it lays out the basic strategy I used to make the creation of a specific instances of my rubric more-or-less effortless.

Next time, I’ll explain how I’ve set up the calculations.

Grading with Tinderbox

Last semester, I spent a few weeks creating an analytic grading rubric using Tinderbox. I don’t generally use rubrics, but they are very popular with some of my colleagues, and I thought I’d see if I could come up with one that could actually work for me and my students.

The rubrics I’d tried in the past were completely non-starters. For me, evaluation should involve plainly written comments about what works, what doesn’t and why. I also want to suggest how things could be improved. Paper rubrics are invariably too constrained and inflexible to support this kind of feedback.

Software tools seem as if they should provide the consistency of a rubric and the flexibility of individual comments. Yet, the systems I’ve tried—and I’ve tried both ad hoc and commercial options—have all been as unwieldy as the worst of the paper rubrics. Getting my comments out of the systems so I can hand them back to students is also too often a chore.

Whether I’ve come up with something better using Tinderbox is an open question, but I do have a working rubric and have learned a lot putting it together. So I’ve decided to explain how it works.

An Overview of the Rubric

To get started I think it would be useful to show the rubric in use, and so I’ve put together a demonstration file that is a stripped down version of my course planning project. It contains the rubric, a student note (my name, fake ID number) and nothing else. In the video below, I use this demonstration file to walk through the process of creating an evaluation sheet for the student’s essay, selecting the appropriate descriptions of their work for the various criteria and then reviewing the comment sheet that results.

I’m fairly certain that all of this will look fairly boring. This is a good thing in practice—a rubric that is fireworks will be a distraction and a lot of the utility of what I’ve made comes from its flexibility which I don’t really show—but alas, demure adaptability makes for bad video. The excitement will come (maybe?) in subsequent posts when I explain how I use rules, agents, and export templates to make everything work. For now though, you’ll just see the following:

  1. A note is created and then automatically converted to a rubric container and renamed.
  2. The rubric container is expanded to reveal the evaluation criteria containers. For each criteria, one child is selected as a comment by ticking it’s checkbox. In one case, the criteria grade is revised manually and the criteria titles are updated as a result.
  3. The main rubric’s key attributes now show the final grade for the assignment. The hover expression will also show the final grade.
  4. A general comment is added for inclusion on the comment sheet.
  5. The preview pane shows the comment sheet that will be exported. Both the general and criteria-based comments are organized using additional, organizational text. Everything has been properly formatted.
  6. The selection for the first criteria is changed and revised. The changes are reflected immediately in the comment sheet.

(There is no sound with the video. The silence feels weird and peaceful I think.)

What I Wanted

My rubric isn’t fancy, but it uses techniques that are basic enough for me to understand and are exposed enough for me to adapt on the fly. I’m sure there are more elegant ways to do some of what I’ve done. That said, what I built does what I hoped it would in the four areas that mattered the most to me.

First, I wanted to select comments not number grades. I did, however, want the rubric to translate my selections into numerical point values. I also wanted it to compile these values into a grade breakdown and to use them to calculate the overall grade for the assignment. I wanted both of these things to happen automatically based only upon the text I selected.

Second, I intended the comments I selected while grading to be given back to students as part of their feedback, and so, I wanted my rubric to be flexible enough to allow me to revise or add to the prepared text in order to respond specifically to individual students’ work. I also wanted to be able to deviate from the basic grade breakdowns when necessary.

Third, once the grading was done, I wanted my rubric to export document files containing comment sheets that I could return to students either electronically or in hard copy. I wanted these files 1) to compile the comments I’d selected and written and 2) to organize and format them in a useful way. I also wanted the resulting files to be named so as to allow bulk upload to my college’s course management system.

Finally, I wanted the rubric to be able to travel from semester to semester. In other words, I needed to be able to revise the criteria, descriptors and grade calculation quickly and easily in order to adapt the rubric to new assignments and new classes. If I had to dive under-the-hood regularly, I wouldn’t do it and the whole project would be a waste. I know myself.

The Plan for Posts

Over the next week or so, I’m going to post a short series that explains how my rubric works. I think I can do everything in four posts, so my planned progression is:

If I need to break one of these posts apart to have more room, I’ll update this list to account for the change. That way it can serve as a table of contents.

Finally, before moving on, I want to acknowledge that Mark Anderson’s TbRef and his responses to questions on the Tinderbox user forum made the difference between success and failure. Mark Bernstein’s “Actions and Dashboards,” which can be found in the Tinderbox Help menu, was my go-to resource for generating ideas and solving problems. I also relied heavily on other people’s questions and posts at the Tinderbox user forums and the backstage group. So thanks to all.

Next post: setting up the prototypes.