What AI assistants can and can't do

Why it matters

AI assistants (chatbots built on large language models, such as ChatGPT, Claude, Gemini and Copilot) are now part of everyday technical work. They draft emails, explain error messages, suggest code and summarise documents. They can save a lot of time. They can also state wrong things with complete confidence. People who get the most from them know what the tools are good at, where they fail, and how to check the output.

The core idea: very good prediction, not a database of facts

A large language model (LLM) is trained on a huge amount of text. It learns statistical patterns in language and uses them to predict, one small piece of text (a token) at a time, what is likely to come next. Fine-tuning then teaches it to follow instructions and respond helpfully.

This explains both its strengths and its weaknesses:

• It is excellent at language tasks: rewording, summarising, explaining, translating, drafting, and producing text in a requested format.
• It does not look facts up by default. It generates what sounds right. When the right answer was rare or absent in its training, it may produce a fluent, plausible and wrong answer. This is often called a hallucination.

Some assistants can search the web or read files you give them, which helps, but they can still misread or misquote sources.

What they are good at

• Explaining a concept in simpler words, or at a different level.
• Explaining what an error message usually means and suggesting things to check.
• Drafting: emails, outlines, documentation, test cases, starter code.
• Reformatting: turning notes into a table, converting units of text into a checklist.
• Brainstorming options you can then evaluate.

Where they go wrong

• Invented details: made-up function names, settings, citations, statistics, legal clauses or paper titles.
• Out-of-date knowledge: models have a training cut-off date. New versions of tools, standards and prices may be missing.
• Maths and counting: they can slip on arithmetic, units and long calculations unless they use a calculator or code tool.
• Agreeing with you: a leading question ("this is right, isn't it?") often gets agreement.
• Safety-critical advice: electrical work, lab procedures, medication, structural loads, legal requirements. Wrong answers here can hurt people.

Rules for using AI at work and in study

1. You are responsible for what you use. "The AI said so" is not an excuse for a wrong number, insecure code or a false claim.
2. Verify against a trusted source: official documentation, a standard, a textbook, your instructor, or a test you run yourself.
3. Protect data. Do not paste passwords, API keys, customer or patient data, unpublished research, or confidential code into AI tools unless your organisation has approved that tool for that data.
4. Follow the rules where you are. Schools and employers have AI policies; some courses and exams forbid AI help. Be open about how you used it when asked.
5. Use it to learn, not to skip learning. Ask it to explain why, then try the next problem yourself.

Worked example: checking an AI answer

You ask an assistant: "In Excel, what does XLOOKUP return if no match is found?"

It replies: "XLOOKUP returns 0 when no match is found."

How to check:

1. Find the primary source. Search XLOOKUP site:support.microsoft.com and open Microsoft's documentation for the function.
2. Compare. The documentation explains that if no match is found, XLOOKUP returns the #N/A error, unless you supply the optional if_not_found argument, in which case it returns that value instead.
3. Test it. In a spreadsheet:

A1:A3 = apple, banana, cherry      B1:B3 = 1, 2, 3
=XLOOKUP("grape", A1:A3, B1:B3)              ->  #N/A
=XLOOKUP("grape", A1:A3, B1:B3, "not found")  ->  not found

4. Conclusion: the AI's answer was wrong. Note the correct behaviour and the source.

The same habit works for code: read the suggestion, check unfamiliar functions in the official docs, and run it on a safe test before trusting it.

Common mistakes

• Treating an AI answer as a source to cite. Cite the documentation or paper you verified instead, and check that any references the AI gives actually exist.
• Pasting confidential data into a public tool.
• Copying code you do not understand into a real project.
• Asking leading questions and taking agreement as confirmation.
• Letting AI do all of your practice exercises, then struggling in an interview or on the job.

Quick recap

• LLMs predict likely text; they are strong at language tasks and can be confidently wrong.
• Watch for invented details, out-of-date facts, maths slips and agreement with leading questions.
• Verify with primary sources or your own test; protect confidential data; follow local AI policies.
• Use AI to speed up learning, not to replace it.

Try it

Open the Do tab to check a real AI answer against a trusted source.