By: Jill Heinze, Research Director at WillowTree, and Matthew Ismail, Editor in Chief of the Charleston Briefings and a Conference Director

Matthew: For years now, I’ve been telling librarians and people in publishing that artificial intelligence is going to transform their professions–and potentially in ways that will require them either to retrain or to find another profession, entirely. I may be wrong, of course–writing a book about a Victorian Egyptologist doesn’t make me an expert in technology–but my intuition is that AI is going to transform relations between people and information–and ChatGPT is just an eye-catching suggestion of that. 

I also believe that researchers will respond enthusiastically to inventive and energetic entrepreneurs outside of academia who are unencumbered by institutional inertia, laser-focused on the needs of their customers, and whose AI-powered tools will give researchers a competitive advantage. Researchers will be willing to subscribe to a resource that is priced for individuals even without the support of the library or the university, thus leaving the latter out of the loop.

It is also true that, for the most part, people I have known in academe just want AI to go away. They variously ignore it, get angry about it, try to have it banned, make outraged speeches about it, insist that it’s biased and therefore bad, say it will never replace a human, etc…But AI is not going to disappear because some articulate people complain about it–any more than it will be successful because a few marketers hype it. When AI-powered tools make research better, then adoption will follow.

And this is why I asked Jill Heinze to join me for this column. I wanted to hear from someone situated in both the tech world and the information sphere how GenAI is changing her daily work. In a digital product development company such as WillowTree, there is no question of making speeches about LLMs but not actually using them. There is a potential advantage in the tech which the company will not cede to competitors, and Jill has been engaging creatively and energetically with LLMs such as ChatGPT as a result.

Jill is Research Director at WillowTree. She majored in history in college and worked broadly in user experience design in an academic setting, so her perspective is valuable for us non-specialists. We had spoken about how GenAI has had a dramatic impact on her work, and I suggested that we might produce an article that illustrates the impact of GenAI on someone who is not trained in computer science and artificial intelligence.

While recognizing that ChatGPT is only one LLM from one company with a particular marketing and product development goal, it’s clear that a big, discontinuous, explosion occurred when ChatGPT launched into the public sphere at the end of 2019. AI research is not new, of course, going back at least to the 1950s, but with ChatGPT the average user was suddenly able to experience Generative AI, which had, until recently, been prohibitively expensive to operate and thus hidden from the public. 

OpenAI also made its GPT API available to developers (https://openai.com/product). As a result, it was incredibly easy to integrate generative AI features into existing interfaces– and Big Tech did just that. While other companies, such as Facebook and Google, had been sitting on their own LLMs for some time, once they saw the advantage gained by OpenAI, they raced to introduce their own products into the market. Microsoft, meanwhile, which is an OpenAI investor, enhanced its Bing search engine and introduced its image generator, etc.

Jill: I remember how I felt the first time I tried ChatGPT. I had the sense that everything was about to change for me. The output, while it wasn’t perfect, was GOOD. And it was easy to see that, after time and refinement, it would become VERY GOOD. 

At the same time, of course, I also experienced a sense of existential dread. I watched the AI Dilemma (https://www.youtube.com/watch?v=xoVJKj8lcNQ) and began reading about Generative AI every chance I got. (Remember, there are decades to catch up on…). I knew my kids would grow up in a world in which human and artificial outputs would be even more commingled, and information literacy, already a tough thing to achieve, would become even more fraught.

I don’t think I’m exaggerating. Generative AI is a form of AI that, as its name suggests, generates outputs from training data. The job it does has been, until now, more or less the purview of human actors. We’re both ceding control of outputs to software and proliferating those outputs within systems and networks (human and otherwise) in ways we can’t predict. And we know that LLMs make mistakes and the data on which they are trained is biased. So, I felt the impact very personally.

Professionally, I’m already primed to think about human experiences and impacts. Everything from my History degree, my experiences as an academic librarian, and my work gathering user insights for software products, makes me attuned to thinking about how people engage with information and each other. Humanities and Social Science folks, in my view, have found their moment, asking important questions about ethics, social justice, philosophy, etc., in relation to AI.

To be honest, my first responses towards GenAI were primarily skepticism and fear. But the more I’ve learned about it the more equipped I am to grapple with the dangers I see–at least at some level. And I definitely see more positives now, as well. Just the other day I was using Excel to enter a bit of data. I was rearranging, breaking up, and reformatting cell content, when I got a help message saying something to the effect of: “I think I know what you’re trying to do, and I have a formula that can do it.” Sure enough, the formula worked like a charm and it would have taken me a long time to look it up and figure it out.

So that’s the personal side. Professionally, I work for a digital agency. Our clients quickly started coming to us with questions about AI and how they can get help deploying it. There’s a lot of pressure from stakeholders at the moment to start experimenting with AI-enhanced products and to get them into production. At WillowTree, we have the skills and data scientists to help. We reorganized our business to solidify this expertise in data and AI (https://www.willowtreeapps.com/services/ai-ml-consulting). WillowTree is consulting with businesses, for example, on how to achieve their goals and user outcomes with AI, but we have also found efficiencies in our own work. WillowTree has a private GPT instance in which the data isn’t shared back to the OpenAI dataset. Our researchers have used GPT Excel add-ons to analyze qualitative data to good effect (after including some good prompt engineering to train the model). I definitely use GPT to help ideate, smooth out writing, and to start exploring topics.  

At the same time, my early experiences demonstrated that we need to be very careful in how we execute AI. When I work with clients and our internal teams, I take care to ensure that we’re all practicing Responsible AI. I think the unique generative attributes of GenAI in particular make Responsible AI very important since it’s also exceptionally risky. 

What might all of this mean for scholarly publishing?

In spite of the offputting hype, and while many people may wish GenAI would just go away, the basic realities of incorporating GenAI into products have been quickly realized. They are here to stay. And GenAI is a particularly demanding form of AI. It poses more risk, requires more data, and needs more maintenance than other AI models. So I suspect that, over time, organizations will be more discerning about when and how they deploy it. But GenAI is already really good at solving certain problems, and has been for some time. Routine tasks for which lots of data for training exist are ripe for the picking. 

In medicine, for instance, Radiology has long been using AI tools to detect anomalies in scans. Call center interactions are already well integrated with GenAI. I think we’ll eventually see GenAI for what it is – one tool in the tech toolkit but not the only or best one for every use case or every organization (depending on resources, risk tolerance, etc.)

Scholarly publishing may think that it will not be subject to these forces, but it will be affected just like other industries where repetition and summation are involved. We know that some scholars have long been gaming the publish-or-perish system, and perhaps AI can help ferret out bogus works and aid the peer-review process. But given the ease with which GenAI can facilitate the process of writing and thematic analysis, scholars may find themselves pressed to push deeper into new-knowledge generation. AI can be a boon to this: Tools like ChatGPT have quite a corpus to interrogate and creatively illuminate hidden relationships across disciplines, time, scholars, datasets, etc. They won’t be perfect but successful users of these tools will push more creative boundaries.

When it comes to “successful” users, I think we’ll see some important skills emerge. We’ll have to be very data-savvy and be able to deeply critique the underlying data from which AIs generate information. As powerful as ChatGPT is, if it continues recycling its own data, and/or disincentivizing online publishers from sharing their data so as to avoid becoming part of training data, we could see a lot of deterioration in quality (https://www.platformer.news/p/the-ai-is-eating-itself). 

Models that operate as “black boxes” will be particularly problematic, and we’ll see more and more specialized datasets being used to fine-tune results for more domain-specific AIs. I think savvy use will also require that scholars understand a bit about how these tools work under the hood. They’re not magic. LLMs are simply using statistics to predict the next most likely “token.” When pushing boundaries in scholarship, simply surfacing the predictable won’t do. They’ll have to really question, even more so than before, the right tools to use to get at novel relationships and insights.

I also think that the notion that we are going to need huge numbers of prompt engineers is overblown. These GenAI models “learn” over time. They are designed to interpret and generate human language (or languages such as computer code) in a naturalistic way. Perhaps in the very short term, prompt engineering will have its day, but I don’t expect you will need any additional expertise to use LLMs other than a good command of language and knowing how to ask good questions. My work as a user researcher has involved learning to ask non-leading, least-biased questions, and this has helped me a great deal in using ChatGPT.

My job in 2-3 years

As always, I don’t think the biggest challenge of technology is the tech itself. I think the bigger challenges involve how we use technology to improve people’s lives at scale. GenAI has really created a sense of urgency for me in this regard. Now is the time to have difficult conversations and make responsible decisions for the future. 

It’s clear that, for every awesome use for GenAI, I can think of an equally terrible use–enhancing education, for example, vs. creating insidiously sneaky political deepfakes. My own job, now, is to be much more educated about the tradeoffs between and within AI models and to contribute to strategic product decisions that are responsible and humane. We now need to make room for engineers and other project team members in ethical and social discussions when we consider the future of the company. Technical expertise should be allied to ethics and social responsibility.

We also need to be more mindful about where tech and society intersect, so I hope to do more work helping colleagues and others to think through this connection. I’ll definitely be using AI-enhanced tools, as will everyone else, so there will be a new baseline. But, professionally, we’ll all have to make sure that as tasks are expedited, we’re not glossing over details or skipping steps, but that we figure out how tools make us better critical thinkers.