Thursday, August 7, 2014

How Not To Do User Testing

  • Perform tests only after a final product has already been rolled out
  • Use your tests to reify assumptions already built into the product
  • Test once and then never again because hey, you’re finished
  • Refuse to accept the validity of any given test until a statistically representative sample of your user populace has been obtained (it’ll never happen)
  • Never change your testing tasks and procedures, even the ones that prove to be deeply flawed, poorly worded, uninformative
  • Ask users for their opinions rather than observing what they actually do. “Do you like this background gradient?” is a particularly apt question.
  • Conversely, test only tasks you think are important without gauging what users think is important
  • Collect personal information and video recordings during tests with no plans for how to secure the data or when to delete it
  • Simply refuse to do user testing

Monday, April 21, 2014

Looping Over Regular Expressions in JavaScript

Much as JavaScript has literal forms for strings & numbers, it also has a literal Regular Expression (henceforth regex) form. So you can wrap characters in single or double quotes to make a string literal, & you can wrap characters in forward slashes ("/") to make a regex literal.

So regexes are literals in JavaScript…except JavaScript is sort of a broken language with regard to literals. The typeof operator is nearly useless.
typeof /foo/
// returns "object"…damn you, typeof
/foo/ instanceof RegExp
// true! hurrah for instanceof
Because regexes have a literal form of sorts, you can do nice things with them like put them in an array & loop over the array:
var tests = [/Foo/i, /BAR/, /baz/i],
    str = 'This is a sentence. Foo, says the sentence.';

// check if each regex has a match in sentence
tests.forEach(function(re) {
    if (re.test(str)) {
        console.log(re + ' is a match!');
    }
});
This is the approach I use in my Wordpress Spam Clicker bookmarklet: I have an array of regexes matching known spammer patterns which I loop over, testing each comment against them.

BUT what if you want to slightly modify each regex in an array? For instance, what if you want to loop over regexes but test for strings with a space at the beginning the match? You can save some typing & potentially (depending on how big the array of regexes is) a lot of bytes by storing a truncated version of the regexes & then modifying them later. Except it doesn't work:
var tests = [/Foo/i, /BAR/, /baz/i],
    str = 'This is a sentence. Foo, says the sentence.';

// check if each regex has a match in sentence
tests.forEach(function(re) {
    if ((/\s/ + re).test(str)) {
        console.log(re + ' is a match!');
    }
});
I'm trying to take each regex & prepend the special character for a space ("\s"), so /foo/i should become /\sfoo/i. But the addition operator doesn't work here, JavaScript doesn't know how to add 2 regular expressions, instead it casts them to a string (typeof (/\s/ + /foo/i) === 'string'). What do?

Well, JavaScript also has constructor functions for all its literals: String(), Number(), & RegExp(). Generally, you do not use these. I repeat, if you're writing code like var count = new Number(0) you can stop it, stop it right now. One reason is that typeof count will return "object" if count was created with a constructor. But also it's just an unnecessary amount of typing.

BUT it turns out that compiling regexes from strings can be done using the RegExp constructor. So to achieve my earlier goal I can write:
var tests = ['Foo', 'BAR', 'baz'],
    str = 'This is a sentence. Foo, says the sentence.';

// check if each regex has a match in sentence
tests.forEach(function(re) {
    if (RegExp('\\s' + re).test(str)) {
        console.log(re + ' is a match!');
    }
});
Instead of storing regexes which are later cast to strings, I can store strings & then essentially cast them to regexes using the RegExp constructor. I have to escape the backslash to ensure \s makes it into the regex, but it works. The RegExp constructor takes the regex flags as a second argument too, so I could write RegExp('\\s' + re, 'i') to make all my regexes case insensitive. This, too, could be very handy & save a lot of bytes/typing.

Sunday, March 30, 2014

Git Tools

On a recent commute I mulled over the various tools I use to make git, the popular distributed version control software, easier to use and more powerful. Here's a round up of what I use, or find interesting.

gitconfig - anything you do with the git config command can also be placed in a .gitconfig runtime configuration file in your home directory. My main use case is aliases that save me a ton of typing. Simple shortcuts like "c = commit -m" are an obvious starting point. But git also has several commands with multiple, handy flags; for instance, my "sweet-looking but concise logs" alias is "l = log --pretty=oneline -n 20 --graph --abbrev-commit". I do not want to memorize and type that monster, not once, not ever. My entire .gitconfig can be found in my dotfiles repo.

gitsh - an interactive shell for git. If you're running a bunch of git commands in a row, enter gitsh & run them without typing "git" over & over. This can be very useful as git commands tend to come in waves; "oh I need to commit these final changes, rebase, checkout master, & then merge this feature branch" is a common workflow, for example. Gitsh also displays repository information—the current branch & working directory status—in its prompt.

hub - a command-line tool for interacting with GitHub. I don't use this because it wouldn't gain me a whole lot of efficiency but I bet hub would be invaluable if you're an active GitHub user.

js-git - an interesting project to implement git in client-side JavaScript with support for various browser storage APIs. Could feasibly bring git into environments like Chromebooks, where one doesn't have command-line access but could still benefit from version control.

Sublime Text Packages

I use Sublime Text as my main editor & these two packages are great in terms of git integration.

Git - this package is essential if you're working in Sublime Text. It gives access to all the common git commands—add, commit, diff, log—right in the command palette. You never have to leave your editor to access version control, you can stay in a single context & do everything you need. It's a huge boon to productivity. I use "Quick Commit" (adds and commits the file I'm currently viewing) all the time. I bet roughly a third of my commits are through that single convenience method.

GitGutter - highlights lines that have been changed, added, or deleted in the file you're viewing with coloring in the gutter. You can select from a few different styles of coloring. This is a small nicety most of the time but can be of great assistance when returning to a project that has a dirty working directory or stashed changes which you've forgotten.

Thursday, March 20, 2014

Start-Up Thinking Is Inappropriate for Libraries

tl;dr — if you believe your institution is a social necessity, start-up thinking is a terrible approach.

A recent conversation with a friend who has worked in the start-up space brought up Brian Mathew's "Think Like a Start-Up" white paper and some unresolved issues I have with it, never publicly articulated. See also: The Marketing Unproblem of Libraries.

#Fail


Most start-ups fail. Start-ups are praised for their agility, their ability to solve problems, but not for their longevity. If you believe in the worth of libraries as institutions, I'm guessing you don't want 75% of them to go under. It's unfathomably, eye-rollingly ironic that Mathews starts his white paper with doomsaying about the sustainability of academic libraries and then offers transient organizations as a model for survival. I can't even.

Trying to flip this fact later in the white paper does little to assuage my concerns. Noting the failure-prone nature of start-ups is not simply some snarky observation; it speaks to irreconcilable differences between how start-ups are run & how are libraries should be run. If you want your library to be around next year, next decade, next century, you probably don't want to emphasize risk-taking. Long-term thinking might be more suitable. You probably don't want to be a technological solutionist. Heck, you probably don't want to rely on the assumption that you only need to serve a population with access to certain technologies. Making an iPhone app is not enough. Making any app is not enough. Being a community-driven organization just might be enough.

It's also worth mentioning start-up culture has its own atrocities. It's hostile to women.* It's hostile to people of color. They're just generally not the type of organization socially conscious people probably want to work for, not that there aren't exceptions to this generality. I find it intolerable to valorize start-up culture while its downsides go unmentioned.

On Choosing Appropriate Proxies


I envision a rejoinder that libraries should praise & emulate the agility & innovativeness of start-ups, focusing on those attributes rather than their ephemerality. Leaving aside the fact that this straw-person argument is basically "but if you only look at the good things start-ups are good," it hints that start-ups are a poor proxy for what we actually want to talk about. I despise poor proxies. They muddle the debate & obscure the underlying issues. To use my favorite example: when we use age as a proxy for technical savvy, we not only discriminate against older folks but overestimate the abilities of the young. So let's discuss "libraries should be agile & innovative," not "libraries should think like start-ups."

But that's a lame tag-line right? And tag-lines are important. It's catchy, "Think like a Start-Up." But if it's so misleading as to be positively counterproductive, it should be ditched.

Exeunt


Finally, there's perhaps a tension in that start-ups are capitalist institutions par excellence & modern libraries** typically follow a more socialist, resource-sharing approach. But that's too much to go into here & I haven't thought about it enough.

In general, there are virtually no similarities between what libraries should be(come) & what start-ups are. Mic drop.

Notes


* There are numerous examples or articles I could have linked to here but Ashe Dryden's is particularly apt. If you think this statement is contestable, leave a comment & I can cite additional instances of hostility.

** Obviously "social libraries" like Benjamin Franklin's Library Company of Philadelphia (unnecessary emphasis mine), where only subscribed members could access the collection, aren't following a very socialist model. These are less common in America today than tax-funded public libraries, for instance.

Friday, January 31, 2014

Open Letter to Middle States Commission on Higher Education

Middle States, one of the major higher education accrediting bodies, is seeking feedback on a new set of Characteristics for Excellence [pdf] in Higher Education. They have a survey which is open for comment but only until the end of today (1/31/14) so I encourage everyone to read the draft and submit feedback. For reference, it may help to read the previous Characteristics of Excellence though they're a lot longer and more convoluted, IMHO.

tl;dr

Accrediting standards for libraries should be more rigorous, certainly not entirely absent.
Also, stop making assumptions about why students attend higher education institutions.

Below are the survey questions and my responses to them.


5. Provide any general comments on the draft of the Characteristics of Excellence (MSCHE accreditation standards):

There's a glaring lack of consideration for libraries, information literacy, and library services in the draft. Specific weaknesses will be addressed in the answers below.
I do want to say that I appreciate the authors' focus on brevity. Whatever my complaints below, this draft is far easier to read, understand, and reason about. This is not only due to its conciseness but also due to the reduced redundancies: no longer must one constantly cross-reference between standards when investigating a single topic, such as assessment of student learning outcomes. It is commendable that this was clearly a focus of the authors.

There is a reason that every higher education institution in America has a library in some form or another, but if institutions were held to these draft standards a library would be an unnecessary expense. Hopefully in my following answers it will become clear why higher education institutions have always had and continue to need libraries.

6. Provide specific comments about the ability of the revised accreditation standards to honor the diversity of institutional mission:


This passage from Standard IV makes assumptions about the reasons why students attend institutions: "the successful achievement of students’ educational goals including degree completion, transfer to other institutions, and post-completion placement". While those are only examples, they reduce education entirely to credentialing (degrees) and job placement. This neglects civic duties like preparing students to be informed, critical, and engaged citizens but also many other educational missions (lifelong learning, job promotion, understanding others, bettering one's self, curiosity, entertainment even). Really, it should either read "the successful achievement of students’ educational goals" with no examples that make damaging assumptions about why students attend the institutions that they do or encompass a far broader range of educational missions. Isn't it enough that institutions support students' goals, not what they think students' goals should be?

7. Provide specific comments regarding the ability of the revised accreditation standards to measure and demonstrate academic rigor and institutional quality:


Standard III #5 which outlines a general education program does not include information literacy, which was covered in the past standards. I would hardly call education which doesn't include information literacy rigorous or quality. While the draft's authors perhaps think that critical analysis and technological competency encompass information literacy, the discipline exceeds those two in places. For instance, critical reasoning does not cover efficiently accessing information, incorporating it into one's knowledge base, employing it to accomplish a specific purpose, or understanding its surrounding ethical/legal/technical issues in the same way that, say, the Association of College and Research Libraries' information literacy standards do. If anything, information literacy is a prerequisite for any critical analysis and more worthy of inclusion. It would be difficult to critically analyze sources when staying within the prescribed arena of assigned readings and one's own filter bubble online, for instance, yet the draft standards do not assure that students will have the means of identifying, seeking, and finding information outside of those areas. Similarly, technological competence doesn't extend to retrieving documents from information systems or ethical inquiry into the innate bias of different technologies and how that bias shapes the availability of information. To mention filter bubbles again, one can be perfectly "competent" at Google searching without realizing that it serves different results to different users depending on a variety of factors such as geographic location, gender, and the web browser being used.

Libraries also provide access to scholarly resources. With no institutional obligation to provide a library to its students, the quality of information available to students cannot be validated. Even if instructors were excellent and the student experience sublime, the scholarly materials available would be lacking. While an increasing amount of academic material is available freely on the web, the vast majority of scholarly literature is still locked in subscription databases. Furthermore, academic information on the web is scattered, difficult to find, and hidden amongst sources of more dubious quality. Couple that with the fact that you have not required that students be information literate and there is simply no hope that they will learn to read and use high-quality information, an effect that in turn reduces institutional quality. Finally, librarians are trained and strive to provide information from multiple paradigms. Without them, academia becomes an exercise in confirmation bias; there's no assurance that students or even faculty will seek out, or have available to them, alternate points of views.

Probably for the reasons outlined above, many other major higher education associations recognize information literacy as a key competency, e.g.:
AACU "LEAP" Essential Learning Outcomes: http://www.aacu.org/leap/documents/EssentialOutcomes_Chart.pdf
New England Association of Schools and Colleges Commission on Institutions of Higher Education, Standard 4 "The Academic Program"

8. Please provide specific comments related to the ability of the revised accreditation standards to measure the quality of the student experience - both within and outside of the classroom:

Standard IV #6 mentions review of student support services "designed, delivered, or assessed by third-party providers" but does not apply the same to in-house services. Apparently only outsourced services need to be of sufficient quality? Or is the implication that support services should not be developed locally?

Further, libraries are essential to student experience and go completely unmentioned in the draft. Libraries provide space for study, whether in collaborative groups or in quiet isolation, as well as territories for intellectual exploration. These territories are increasingly digital; do not picture simply a student roaming tall shelves filled with volumes, but also browsing interactive digital archives that serve both to sustain cultural memory and stimulate curiosity. What's more, many libraries are engaged in creative endeavors that involve facilitating student production of various artifacts, whether those be videos, podcasts, publications both print and online, or artifacts produced by three-dimensional printers.

It is not merely that libraries go mentioned which is disconcerting, but that Middle States has never sufficiently assessed libraries. I am currently in the middle of a self-study and am personally disappointed at how meek the library requirements are. The standards seem to ask "Do you have a library? If so, check yes." They do not ask that library services be responsive to student needs and assessed for their efficacy. Holding libraries to the very low standard of mere existence damages both the profession of librarianship and higher education at large. If anything, rather than excising all mention of libraries from the Characteristics, your organization should seek more substantive demonstrations of value from libraries.

9. Please provide specific comments related to the ability of the revised accreditation standards to maintain a focus on continuous improvement while demonstrating meaningful institutional outcomes:

(I had nothing to say here, plus was rambling too much elsewhere, so left it blank)

10. Please provide specific comments related to the ability of the revised accreditation standards to encourage and support innovation:


While I see nothing in the standards that specifically encourages innovation, I see much that limits it. Specifically, commitments to specific planning, documentation, and reporting structures limit the agility of institutions, particularly small ones. Innovation is given lip service in Standard III #2 subsection d, which is shared with professional development. If it's so important that you ask for feedback in this survey, perhaps it deserves more prominent focus in the Characteristics.

One improvement might be recognizing the role that failure plays in innovative organizations. Language is powerful and an accrediting body actually acknowledging that failure can be a learning and growing experience would be of immense benefit to higher education. Accreditation has traditionally been a punitive exercise; do something wrong and you are warned or lose accreditation. What if you reframed it as a process that rewards experimentation? Experiments often do not work out, but if results are shared properly then they prevent others from making the same mistakes and increase the likelihood that future efforts will succeed. Encourage effort and sharing as opposed to punishing failure.