What should an AI tutor make easier?

I've been building an AI tutor for French. One of its reading activities asks the learner to explain the main point of a short passage, pick out an important detail, and show which words support their interpretation. The tutor uses activities like this to help judge whether someone is ready to move on.

Here is one passage the product used, translated into English: repairing appliances reduces waste and teaches useful skills; a free workshop is happening on Saturday.

The question seems straightforward. But someone learning French may get stuck on the words for appliances, waste, or workshop before they can say much about the passage. A wrong answer leaves the tutor with a choice: work on understanding an argument, or first work on the words needed to read this one?

I changed part of the tutor's passage selection to take recorded vocabulary into account. Instead of simply rotating through a small set of passages, it could prefer one containing more words already in the learner's record.

For someone with vocabulary about work and commuting, that could mean a passage about working from home two days a week because the trip to the office is long. The learner still has to identify the point, find a detail, and explain the reason given in the text. There is something to figure out, even if more of the words are familiar.

How much can I adapt the question before I need to change what I conclude from the answer?

What does a correct answer show?

If the immediate goal is to practise finding a reason in a passage, familiar vocabulary can give the learner a way into that task. They still have to connect the ideas. Knowing the words for office and journey doesn't by itself explain why someone prefers working at home.

But vocabulary is part of reading too. Outside the tutor, nobody promises that the next paragraph will use words you know. If I want to understand how someone handles unfamiliar material, choosing a comfortable passage removes part of the challenge I need to see them face.

The same adjustment can therefore make sense for one purpose and weaken another. I need to be clear whether the tutor is helping someone practise a skill or checking how far they can use it beyond familiar material.

In this change, the reading criteria stayed in place while the passage selection changed. Some of the passages were simplified too. I can't infer equal difficulty just because the prompt still asks for a main point, a detail, and supporting words. Nor does a word appearing in the learner's record prove they know it well.

The tutor is choosing what it gets to see

An adaptive tutor has an unusual influence over its own evidence. It chooses what to ask, then uses the answer to decide what to ask next.

That can be helpful: the next activity can focus on a gap the previous one exposed. It can also become circular. If the tutor keeps choosing familiar material, good answers may tell it to keep doing exactly that. The learner's record could grow more encouraging while leaving unfamiliar situations largely unexplored.

I don't want to solve that by making every exercise harder. A paragraph full of unknown words can reveal very little about which part of reading needs work. The useful comparison would be whether someone can carry the same skill into a different passage after practising it in a familiar one.

That's what I would want to examine next. After practising with a familiar passage, can the learner still find the main point and explain the reasoning in a less familiar one? Where do they get stuck? Those answers would help me decide whether the familiar material is preparing them to read more widely or just giving them a comfortable place to stay.

For me, this makes personalization a more specific product decision. Before asking the AI to adapt an activity, I need to decide which difficulty it should remove and which difficulty is the reason for doing the activity at all.