The 5 biggest market research trends today

The great informers

I’ve been in the market research industry for 30 years and it’s treated me well. With my move away from running day-to-day MRX agency operations it’s great to have a bit more time to look inwards at our industry again.  I’ve enjoyed learning so much about clients’ markets, but it’s good to see what’s happening closer to home. Thank you Greenbook, Quirks and the MRS for feeding my thirst for knowledge.  I’ve noticed five main changes and trends: 

1. LLMs & Qualitative analysis 
2. Knowledge Management
3. Data and respondent integrity
4. Data models and synthetic audiences
5. Democratization of the research process.

1. LLMs & Qualitative analysis 

Being an industry that collects a great deal of unstructured data LLM analysis, or Large Language Models is a big buzz in research webinars now with lots of MR tools popping up. The models quickly analyze unstructured data and find patterns. 

This of course helps the role of qualitative market researchers, doing much of the heavy lifting. We can now focus on the more skilled and fun parts of asking questions and building relationships with people to understand important topics better.

Before, Qual researchers would spend tens of hours sorting through interviews and transcripts, doing long table analysis to find themes and quotes. Many still do! Now, it only takes about five minutes using a LLM system that even links back to specific quotes in the report.

LLMs have been used in marketing tools for personalized content creation for ages, so it’s good to see that research tech is catching up with martech (marketing technology). So much so that we now have a trendy category name of our own ResTech (I’ve spent years researching FinTech, FemTech and MarTech, but never ResTech!). There are many, many new products available for research purposes. – In my quest to get my head around what’s out there I’ve pulled together a list of some of these I’ll share with you soon.

Personally, I love this particular jump forward as Qual research is my thing and I love the feeling of getting to the useful bits quickly. 

2. Knowledge Management

The world is now awash with data so another development I’m pleased to see is improvements in knowledgement management.  We have got to a place where the quick access of relevant knowledgement could become the most important differentiator between competitors (not just MRAs, but Brands at large).

Prior to 1980 (knowledge management 1.0), finding information was hard because it was all on paper or in computers that were not easy to access. Yes, I remember starting work and having to use floppy disks and visit libraries.

After that, in the mid 2000s (knowledge management 2.0) we could access information via SaaS services and search engines like Google helped us find information faster . We could create links to pertinent elements of the document to be referenced later. Much of the content was still human created and machines could only answer simple questions.

Now, we are in knowledge management 3.0 with AI using human-made and some machine generated content to give smart answers to our questions. We’re now able to put things into buckets of data and shift through it to deliver intelligent summaries in response to multifaceted questions. This is a game changer for research analysis as it can produce summaries against research briefs. Not only is that time saving but great for output quality as it’s amazing how often Execs forget to double check their recommendations against research objectives.

With continued training, the tech becomes smarter at helping us.

We are close to reaching knowledge management 4.0, where AI can think more like humans and help draw conclusions by asking the right questions in different ways. It can use internet databases, e.g. behavioral data, social media, human created documents. It can not only do the diagnostics but modeling and predictive analysis. 

Nowadays, researchers need strong critical thinking skills and curiosity to keep up with these changes.  They need to become ever evolving Prompt Engineers. The technology will make insight work easier, faster, and more impactful but also require us to raise their game.

3. Data and respondent integrity

With the sea of data available, data integrity remains an issue. For insight professionals collecting original data from online platform surveys, respondent validity can be challenging due to “ghost completes.” These false responses can be found and removed by quality review processes. Agencies like Sapio Research reject around 10% of all the survey responses it receives and unfortunately it’s seeing the number of ghost or false completes as the bots become increasingly active.

I don’t think data quality (like leadership skills) is talked about much in the research industry compared to other topics. People want to do good quality work, but don’t want to shake the apple cart and put a spotlight on potential issues. But without reliable data, insights are a farce. Fortunately there is new technology coming out that helps with getting more accurate audiences and finding bad data. (Some of these will be listed in the spreadsheet I’ll share soon).

However, we must remember that sometimes we remove unusual data points to make our results look cleaner, or more like we expect the results to be. But by doing this, we might miss important new trends. Reviewing data well is very important.

4. Data models and synthetic audiences

Not only is the quality of data concerning, the availability of respondents is becoming more difficult. People are busier now, they want higher levels of payment, especially in business-to-business research. Respondents have become familiar with more remote research methodologies post covid, so it takes a lot more effort to persuade them to participate in studies online or in person.  Sago’s President, Isaac Rogers has given some really interesting talks in this area. 

A solution to respondent sacristy is using data models and synthetic audience data instead of data captured from asking respondents specific questions. For some time there have been interesting things happening with synthetic audiences outside of the traditional research world in advertising and banking for example.

With the difficulty in respondent engagement we should be saving original research questions for new categories, markets or issues, rather than repeating questions that are already readily available out there. So well trained synthetic audiences have got to become more important. 

Another advantage they bring is that you can train them with counterfacts for example, or ensure that they’re fed with information from traditionally underrepresented groups. This is helpful in overcoming issues associated with under representativeness of some survey panels.

5. Democratization of the research process

I talked to an old friend who works at a big tech company as a UX researcher a few weeks ago. I was amazed by the tools she has for planning projects, designing surveys, hiring people, and analyzing data. She can do the work of three or four researchers from just a few years ago. Amazing.

Most of her time is now spent talking to colleagues, i.e her internal clients, about how her research helps them make decisions and improve their businesses, helping them dig deeper and test their own hypothesis. Unfortunately she’s worried that technology might replace her job in a few years. This idea is both scary and exciting because she can achieve so much with these new tools but then worries about her own worth.

It’s inevitable that these easier to use research tools, and I’m not just talking DIY survey platforms, but design and analysis tools are being used by ‘non researchers’ more frequently. Great for the ReTech market, but concerning for Insights professionals perhaps.

But both the new tools and my friend are helping democratize the use of data by educating colleagues, actually increasing the value of our craft.  The right information helps us make better decisions, that is our purpose.

I can see the need for super users of research tools growing as their functionality increases. We might see a new sub industry growing up to optimize the set up and use of the tools. Look what happened to Salesforce and Hubspot with its partnership programmes!  

Anyway I believe that the best teachers are those with the war wounds and have experience of the coal face, but enough of that for now.

What trends would you add to this list, or disagree with? Drop me a line and help us understand.

If you’d like to hear Jane’s interview on MRX trends look out for Goodman Lanterns Podcast