ChatGPT is impressive out-of-the-box but challenging to apply to real-world problems. Area4 Labs and Hearby are building with AI technology to create a data-driven live event concierge.
The following comes from Hearby, a fast-emerging player in concert discovery and a DMN partner. Enjoy!
Full disclosure: “Train” when applied to Chat technology is the same “train” that we might apply to cats. That is, we ask them to do things they were going to do anyway in a way that doesn’t displease them, and then we figure out how to be happy with what they did.
This has been our biggest lesson in creating “Ask Hearby,” our AI chatbot music concierge. In this article, I’ll bring you behind the scenes on our AI adventure.
At Hearby, we aim to use technology to find and uplift grassroots music and help people find the wonderful music hidden right in their neighborhoods. Whether you’re looking for a night of clubbing, a free classical concert, or music to keep the kid out of your hair, it’s all out there. You may not realize there’s a great music venue right in the industrial park next door, on the dockyard of Liverpool, or in a thrift shop in London.
We want to get people exploring and finding music that they’ll love, and to do this, we spent a lot of time investing in fast search technologies, data-driven filters, and map visualizations. Then we ran right into the wall of ‘Too Much Stuff.’
Enter the chatbot, which allows fans to fast-forward and say what they want without going through all the tedious steps of searching, filtering, and reviewing results. It’s a lot of work for something that should be fun.
However, my experience with chatbots has been a big meh, and we wanted to do something more intriguing.
Our main product requirement was “be useful and don’t be irritating”. Yes, it took us a long time to get that, and we fell off the dragon a few too many times. But seeing how this ubiquitous technology works and encouraging more ideas and dreams with it has been interesting. At its core, it’s a foray into using a Large Language Model (LLM), and I will breathlessly say the possibilities are unlimited.
It took a while, but after several tries, we finally have something useful and entertaining to use. So here’s the behind-the-scenes on what we tried that didn’t work — and what finally did.
-
- Train, train, and train again.
- Give me all the data! More data!
- Hybrids: Just how many technologies can we cram in here?
- It’s a sandwich.
So first, a little more about Training when it comes to Machine Learning.
I need to bring up the topic of training, partly for my snazzy title but also because it’s at the bottom of everything you’re hearing about AI.
To train ML, we first choose a neural net architecture, then give it a vast set of data items labeled with the correct answers (for example, Cat/Dog, T-shirt/Skirt, Pedestrian/Bollard). This type of supervised learning is expensive in computing power, requiring a huge amount of ethically obtained, accurately labeled data. Training enables the ML architecture – the layers and feedback loops that make up the neural net – to adjust to create maximally accurate predictions. For example: “99% chance this image is a cat”.
Going beyond cat/dog to something actually relevant quickly gets expensive and time-consuming. It’s pretty much prohibitive on large data sets for all but the biggest players. Enter LLMs, which come ready-trained on massive amounts of human text right out-of-the-box for anyone to use.
This is what powers our chat dragon: the ability to “understand” human language, figure out what is being asked, and create amazing responses in human language. On the topic of whether there is any actual human-style understanding of concepts, I can start an argument in an empty room (so I won’t go there). It doesn’t matter for our purposes as long as the output is accurate, useful, valuable, safe, and reliable.
This brings me to our challenge: how to make already trained chat technology do what we want.
For a small amount of money and a lot of delight, you can get a subscription to Open AI’s ChatGPT, which will happily write you a letter to Grandma, your term paper, or a pretty decent novel – at least better than anything I can write. Whether soulless or best-selling is in the eye of the beholder, but I prefer to consider it a fantastic tool to help spur creativity.
But as impressive as this is, these out-of-the-box answers are standalone, and the type of chatbot we wanted to create is a conversation that builds as we go along, with context and informality, powered by accurate event, venue, and band data. The challenge, then, is how to get a language-based model to incorporate this external data and use it in its responses and how to have the conversation build as it progresses (memory).
Data! Give me all the data!
The challenge is getting our data into ChatGPT to inform its responses. In a “normal” program, this is a matter of, well, programming. However, an LLM is different: rather than programming, information needs to be text-based to be taken on board.
It’s weird, but not so much when we remember this is a language model. This is precisely how we listen, take in new info, understand it, and use it to inform our actions. In all fairness, the latest models also allow other forms of input, expanding beyond text input. But this is where it was when we started, so that’s where we began.
We started with text-to-sql, in which we describe in words how to find the answers to questions using the tables in our database. So, essentially, telling a programmer how to formulate database queries. This sounded so crazy and improbable that we thought it just might actually work.
Sometimes, it did, but mostly, it sulked, made stuff up, or ignored us. Or all of the above. If you’re thinking cat again, I’m right there with you.
Bring in the hybrids.
So, we moved on to hybridizing and searching our database using ChatGPT for its language capabilities. Among the many challenges:
(1) Knowing what the fan is asking about – An event? A venue? A neighborhood? A genre? A person?
(2) Find the data in our database with a fuzzy search – the whole point of chatting is that the fan doesn’t have to be specific.
(3) Get the data into ChatGPT in words, which is all it understands.
(4) Receive a human-ready answer from ChatGPT.
(5) Augment that answer with links and images.
We quickly realized we needed to confirm it was using our data and not going elsewhere, which in LLM terms is called temperature. Or, in human terms, don’t make stuff up!
It’s a sandwich
After a number of tries, we ended up with a workable sandwich of technologies: Bert NER to understand what the fan is asking about; specialized models to detect essential but idiosyncratic info like informal dates (“in 3 weeks”); a vector database to translate a fuzzy human question into something specific we can ask our already existing search capability; a layer to feed the search answer to ChatGPT in words, and then a method to receive the ChatGPT response in human language. And, finally, a layer to augment it with images and links.
Voila! If this all sounds like a bit much, I get you. But we were delighted to see that a fan can ask a reasonable question, “What’s on in London tonight?” or “Where can I take Aunt Nelly for a jazz brunch?” and get a believable answer that makes sense.
More interesting is that a fan can ask an unreasonable question and get an answer about music events or venues, and an explanation as to why, or, if it’s too far a stretch, simply a reasonable on-topic answer. And, to put your mind at ease somewhat, some questions bring in the guard rails: “I cannot assist you with that”.
Onward!
In addition to our chatbot launching later this year, we are working on several other AI efforts, mainly in Machine Learning and classification. These are focused on highlighting the music scene for fans and encouraging them to explore and find new music and venues. Off their sofas and into venues!
The chatbot has been a very interesting excursion for us into LLMs, which have enormous potential to change how we live with software. So, I hope this has shone a little light on this powerful technology for you.
We’re focused on music and using these incredible tools to uplift grassroots music. Still, I hope this gave you some ideas on how this kind of technology might help in your part of the music world – places where you want people to be able to get to the point faster, have informal access to better information, or be able to explore and expand on an idea on the fly.
