Neural Networks in Marketing Research:
How They Change The Rules of The Game

By now, ChatGPT managed to make a lot of noise. OpenAI has lowered the barrier to entry into AI technologies to the level of "if you know how to use the chat - go ahead, have fun." Users were simultaneously impressed and frightened by the illusion of a deep understanding of whatever the chat was fed, the high speed of processing material of any complexity, and its variability. People instantly realised that AI could easily do the jobs that many of them were paid to do.

Programmers' biggest fear is being replaced by a program. Strong AI has brought this prospect closer: some large language models (LLMs) already write good enough code. Let's use an analogy to understand that researchers are not very different from programmers.

Now let's consider a simple research problem:

The client cannot come up with an algorithm for solving the problem, but they can explain what they need. At this point, you will need another specially trained person - a researcher who will:

a) read the client's problem,
b) come up with a solution algorithm using research methods and tools,
c) provide the client with information for making a business decision in the form of a report.

The researcher is a kind of "interface" for the client's access to research tools. The "interface" accepts a task from the client in the form of text and gives them an answer – again, in text. It turns out that text, language, is the most powerful of researchers' tools. Researchers use language to perform each and every task.

Working in ChatGPT is similar to programming in simple human language. Instead of code, we use prompts - a set of arbitrary instructions about how AI should behave and what it should do. A prompt cannot be specified to just any length, it has to adhere to the limitations of the LLM model. In our experiments (which we will describe below) we used gpt-3.5-turbo and a "smarter" version of gpt-4, as well as free open-source models (gpt4all-j 1.3, BLOOMChat-176B, MPT-7B). The listed LLM models understand complex texts well and provide (seemingly meaningful) answers.

What then is the fundamental indispensability of the human researcher? Let's try to take everything "human" out of the standard research process and take a closer look at where we end up.

This is one of the most painful stages that affects the success of the study. The researcher often needs several working days of long correspondence by email in order to process the initial brief and extract from it a clear statement of the results that the client actually needs.

Here is an example of a fuzzy, unclear request to a research agency:

The phrase "test hypotheses and insights" is particularly unclear here. Testing is about quantitative methods, surveys; but we usually get insights at the qualitative stage, and the client didn't mention anything about that at all. Our manager needed 3 iterations of correspondence to clarify the brief.

How can we get AI to do this job for us? For example, we could write the following prompt:

Like our manager, the neural network figured out what information was missing in the request. More clearly than the manager, it formulated possible situations and expected results. Imagine how the customer experience and speed would change if, instead of reading three letters from the manager, they simply had to choose: do you want to get something similar to No. 1 or No. 2 in the results?

Let's assume that the client said "We want to get No. 1 - the needs of people, the characteristics of their behavior, their attitude towards our idea." We can automate the decision of choosing which methodology to use. Great - so, we feed the model the following prompt:

What did the model say?

This is a good result. But what is qualitative research without audience selection? Let's remember the client's description: "Owners of cats and dogs aged 18 to 65 years. Inside they are divided into: those who already use natural and organic products to feed their pets; those who use standard food, but are interested in healthier alternatives and are willing to consider switching to natural products; those who have not thought about replacing food, but are open to information about the benefits of natural nutrition for their pets."

The approach to audience selection may vary. Typically, this is done through a recruiting questionnaire. We give the model a prompt along with examples of similar questionnaires from our experience:

The model copes with this swiftly and simply:

The model not only set the necessary qualifying conditions and filters but also divided the target audience into segments in accordance with the client's description. You can immediately ask it to provide an answer in xml script format, which we use to prepare the electronic questionnaire. By providing the model with examples from past projects, we get the following:

A trained eye will notice that the model even placed "semantic emphasis" in the wording with tags ([u]...[/u]), that is, it proved to be more caring than the average processor. In general, our toolkit is ready.

ChatGPT is designed to communicate with people, understand their instructions, and generate responses based on them. But it's not so easy to make a "cookie-cutter" model do the opposite – want something from a person. We had to do some heavy lifting to get the model to follow the guide without making the respondents doubt its level of awareness:

How do respondents react to such communication? Is it possible to get a real high-quality answer in this format, like in an in-depth interview? We experimented with different audiences and tasks. Now let's try to communicate with hard-to-reach youth on the topic of sports clothing.

To the open question "In what situations do you wear sportswear?" in a standard questionnaire we get an average of 22 characters per line. Our auto-moderator asks three questions instead of one: the respondent sees the original question plus two "probing" questions, the content of which depends on the respondent's answers. Our experiments show that the quality of the answer with probing increases nonlinearly: respondents write not 3 but 7 (!) times more text - 151 characters on average.

This is achieved through a more human attitude towards the respondents: our auto-moderator is friendly, empathetic, and lives in the chat that everyone is familiar with. Of course, we are still a long way from achieving qualitative data depth.

The correspondence between a respondent and auto-moderator is not exactly light reading. Fortunately, LLM easily copes with the task of creating a summary of the interview so that it turns out something like this:

Searching for insights in this format is much easier. But what if we have a quantitative task? 50 full interviews = approximately 50 pages of unstructured text. A lot of information and work for an experienced coder.

With LLM we will process this ten times faster. One interview with 10 questions is an ideal "chunk" of 1000-1500 words. It successfully fits into a model that picks out individual semantic units from the content:

The last point about "driving in the car" is something that the respondent indirectly mentioned in another part of the transcript:

So, with a simple coding algorithm, we can turn qualitative data into quantitative data. This opens up a path to a new research type that combines all methods' advantages.

Our imagination + LLM capabilities allow us to ponder our dream system for generating insights and - instantly! - their verification. Imagine that you have an interface that runs in human language. You tell it:

We get:

For now, this layout was created by our designers. But the data it presents is real - we got it through a step-by-step interaction with the GPT-3.5 API:

- received a list of answers to the question in the first transcript, wrote down quotes;
- found which answers from the list were in the next transcript, and wrote down the quotes;
- found answers that were not mentioned earlier, wrote them down along with the quotes;
- and so on for each subsequent interview.

This is the most basic version of qualitative-quantitative analysis.

Nothing prevents us from going even further in our thought experiment. Theoretically, the role of the researcher in this process can be reduced to a task setter: "Based on {research data}, give recommendations on what the client should do, guided by the following {action standards}." If these same "action standards" are on the system side, you could remove the "researcher" figure entirely and let the client interact with the data in its raw form. However, by the time such a system is created, we cannot be sure that, on the client side, there will still be a human being to perform the interaction.