Bibs & bobs #9

Delegating research work to machines

Machines of various sorts have always played a role in doing research. I recall from a long time ago and in a universe too distant to remember that I used a piece of software written in Fortran to model experimental NMR data called LAOCOON.

Today we routinely use apps to write with, search databases, model with (spreadsheets and related software) and many more routinely used pieces of software. 

Now with the availability of Large Language Models (LLMs) [1] it was not surprising to see a number of AI apps that are designed to support some research tasks or some that might be exploited to help in routine research work. I have mentioned some of these in previous Bibs & bobs.

Paper digest is one of a number of interesting apps that one might play with to see what it is capable of. David Beer has a useful commentary and account of his explorations with the app. 

My sense is that we will try out and sometimes adopt one or two of these apps if they manage to demonstrate they are better at doing some of the tasks one routinely does in research. Most of these apps make use of GPT-3 and will likely make use of it’s soon to be released successor, GPT-4. Supposedly this may improve the quality of output from the various dependent apps. What is in prospect for GPT-n is anyone’s guess. 

But back to the here and now, the app called Elicit, is at present, pretty handy in many respects. But to me, the area that is much more interesting is indicated in the figure from Erik Brynjolfsson’s paper, The Turing Trap [2]. It is the new tasks that humans can do with the help of machines.  There are many “adjacent possibles” [3] to be explored.

Even at this early stage, the in-between time for AI in research assistance, the odd or weird way in which these models operate, alien intelligence as Rudy Ruggles [4] once put it, can at times, nudge you to seeing dots that can be joined, connections that are new to you at least.

The other spin off from working with machines like those I have mentioned is that that the detail of working with such apps can be easily documented and replicated if necessary. They ought to go to make up a useful component of any research reporting. This is a prospect that is easier if you publish in places that have escaped the A4 mindset.

Delegating work to machines in formal education

This past week the Australian Association for Research in Education held its annual conference in Adelaide. One of the keynotes was given by George Siemens. He spoke about AI as one of the biggest challenges facing education [5]. I fully agree that AI is the biggest challenge but it is also likely to be a significant opportunity and not in the way that is typically argued for in education circles, that of improving things. Simply, things will change and change in ways that are difficult to anticipate and plan for. I’m not heartened by the history of formal education’s engagement with the digital which boils down to: domesticate it and if it can’t be domesticated, ban it. 

LLMs will prove difficult to shoehorn into the domesticate or ban approach. They are developed from text, a lot of text. Having text apps that are predictive, think spellchecker on steroids, generates a lot of intriguing issues for teachers, students, modes of assessment and so on. In relation to this, Richard Gibson wrote a useful post in which he reported his experiments with GPT-3. It mirrors to some degree the opinion piece that Mike Sharples published [6]. Both are well worth reading.

The recent release of GPTchat will likely shake up the educational establishment. How these developments are framed now will play a big part in what happens in the near future. Even now, the current framings fall into familiar patterns to be found in the history of formal education’s engagement with the digital. Drawing on one I developed a long time ago, they can be labelled boosters, doomsters, critics and anti/de-schoolers [7]. I hope that for the new kid on the block, AI, that the labels don’t require too much dot joining. 

I have never found those framings particularly interesting or even useful. For my part, framing these developments holistically is more helpful, i.e. the human plus machine, centaur as some would suggest points to an augmentation, a dependence of one on the other. Framing these developments as changes to the way things are done around here [8] is then a first and important step. Seeing an holistic technology as formalised practice necessarily connects to culture. The impact that search which costs next to nothing has had on how we work, think and carry out routine educational tasks illustrates the point well. The prospect of prediction that costs next to nothing will likely be more profound than that of the cost of search getting close to zero.

My second move draws on the work of Bruno Latour [9]  in which he argues that in any delegation of work to a machine, there is an exchange of capacities, i.e. for a machine to do a task it requires a human to do something new, something additional to the delegation of work, something complementary. A simple example is that of the use of a calculator. If a calculator is used to calculate a sum and the user has no approximation skills then the number provided by the calculator can’t be checked [10]. 

For something like GPTchat two things come to mind. There is a skill to be developed in terms of prompting the app. Secondly, once the app has generated its output there will always be a need for evaluation, something these models are incapable of as they have no context.

AI appropriately framed, can be positioned as an augmentation to what humans do. We all need augmentation. We depend on machines to do so much of our work already, including all sorts of chemical and mechanical augmentations [11]. Some folk are more in need than others, those with disabilities of various kinds. But we are all in need of them, me perhaps more than most. The potential here is enormous. There is much more to say. For now this is where I’ll leave it.

Blogging impact on policy

An interesting blog post from LSE on the citation of LSE blog posts in policy documents. It is not a big impact but appears to be one that is steadily growing. The grey literature rises. 

More on path dependence

Trung Phan has a detailed account of some of the classic examples of path dependence in technology [12]. His newsletter is well worth a follow if you have interests in AI as is Ben Tossell’s.  

A directory of AI-based apps

If you find all of these developments of AI apps somewhat bewildering, it’s useful to recall that the drivers of these apps is venture capital which seems to have bottomless pockets, for now. It reminds me of the time when early microcomputers were the state of art for desktop computing and there was a huge investment, for that era, in software that ran on these devices. Education then and now always identified as the market to crack. 

Futurepedia.io documents all new, popular and verified apps with details about costs and what they supposedly can do. 

Big numbers

When it became clear that the growth of data on the Internet was growing rapidly, helpful explanations in terms of the number books stacked between earth and the Moon and similar illustrations have become common.  The latest [13]:

By the 2030s, the world will generate around a yottabyte of data per year — that’s 10^24 bytes, or the amount that would fit on DVDs stacked all the way to Mars.

Which has prompted the need for new names for these mind numbingly large or small numbers.  I can recall vaguely when I came across petabytes. It was much later than when the name was chosen to indicate 10^15. From the article:

Further from the article,

With the annual volume of data generated globally having already hit zettabytes, informal suggestions for 1027 — including ‘hella’ and ‘bronto’ — were starting to take hold, he says. Google’s unit converter, for example, already tells users that 1,000 yottabytes is 1 hellabyte, and at least one UK government website quotes brontobyte as the correct term.

Just a few prefixes with which to dazzle your colleagues. The paper is worth a skim.

                                                                                             

[1] Venkatesh Rao suggests LLMs should be called memory creatures. His piece on AI as superhistory and mentioned in Bibs & bobs #2 is important. 

[2] Brynjolfsson, E. (2022). The Turing Trap: The Promise & Peril of Human-Like Artificial Intelligence. Daedalus, 151(2), 272-287. https://doi.org/10.1162/daed_a_01915  

[3] A term coined by Stuart Kauffman and popularised by Steven Johnson in his 2010 book, Where Good Ideas Come From.

[4] Ruggles, R. L. (1997). Knowledge management tools. Butterworth-Heinemann, p. 205 

[5] There is a brief account of his presentation here. 

[6] Sharples, M. (2022). Automated Essay Writing: An AIED Opinion. International Journal of Artificial Intelligence in Education, 32(4), 1119-1126. https://doi.org/10.1007/s40593-022-00300-7 

[7] This crude categorisation was something I developed the night before a keynote presentation I gave to a Principals conference in Darwin in 1996. I wrote some scripts for each of the stereotypes and four Principles kindly volunteers to play the parts. They all did brilliant jobs hamming it up! I still have the scripts if interested.

Subsequently, a good colleague and I made use of the categorisation more formally: Bigum, C., & Kenway, J. (1998). New Information Technologies and the Ambiguous Future of Schooling: some possible scenarios. In A. Hargreaves, A. Lieberman, M. Fullan, & D. Hopkins (Eds.), International Handbook of Educational Change (pp. 375-395). Kluwer Academic Publishers. 

[8] A framing Ursula Franklin made, drawing on Kenneth Boulding, in her excellent 2004 book, The Real World of Technology.

[9] Latour, B. (1992). Where are the missing masses? Sociology of a few mundane artifacts. In W. Bijker & J. Law (Eds.), Shaping Technology/Building Society: Studies in Sociological Change (pp. 225-258). MIT Press. http://www.bruno-latour.fr/sites/default/files/50-MISSING-MASSES-GB.pdf  

[10] You could add additional complementary skills to this task, like an understanding of significant figures and perhaps, in precision calculations how the calculator does its arithmetic.

While the complementary skills in using AI apps may appear to be simply checking the generated output, I’d argue that we need to proceed on a case by case basis and identifying what complementary skills and knowledge are necessary is not a trivial task.

[11] I’ve lost count of how many I have. Good thing i have a machine to keep track of some of them.

[12] I pointed to the notion of intellectual path dependence in Bibs & bobs #8.

[13] Gibney, E. (2022). How many yottabytes in a quettabyte? Extreme numbers get new names. Nature. https://doi.org/10.1038/d41586-022-03747-9