Lesson 30 of 32
The Senses of the Machine
Files, Images & Voice
These assistants can now look at a photograph, read a document, work through a sheet of figures and listen to your voice - so stop labouring to describe what you could simply show. But a picture without a purpose is a ship without a port: name what the thing is, supply the context the camera could not capture, point to the exact page or column that matters, and say what you want out of it.
This is Gemini's home ground, and the chapter was written for it. A photograph straight from the camera, a PDF, a sheet of figures - all go into the chat, and if the document already lives in Google Drive you can point at it there instead of uploading a second copy. Gemini Live is the voice half: open it and you are in a spoken conversation, in English, Hindi, Tamil, Telugu and many more, and you can hold the phone up to the thing you are asking about. The step the others cannot match is what happens afterwards - the checklist, the summary, the table goes out to Google Docs or Sheets, so a chat answer becomes a document you can print, share or keep. And where the question turns on something current - a disease season, a tariff, whether a circular still stands - ground it in Search and read the link before you believe it.
From the book Upload a photo or file in the chat, or open Gemini Live to talk by voice instead of typing. If your documents are in Google Drive, you can let Gemini work with them, and export the answer to Google Docs to keep it.
- Attach from the camera, or point at the file already in Drive.
- Say “only the attached file, do not search” for your own documents.
- Ground changing facts - season, tariff, circular - and open the links.
- Export the finished checklist or table to Docs or Sheets.
- Use Gemini Live to ask while holding the phone to the thing itself.
Instead of
What is wrong with this leaf?
Gemini version
[Attach the photo of the leaf, one of the underside, and one of the whole plant.] I grow tomatoes near Kolar, Karnataka. The monsoon has just set in. About 20 of my 300 plants show this - brown spots with yellow edges, mostly on the older lower leaves, spreading over the past week. From the photos alone, first: describe what you see, and say if the image is not sharp enough to judge. Then ground this in Search before you name anything. Which tomato diseases are being reported in Karnataka this season, and what do the University of Agricultural Sciences Bengaluru or the Karnataka agriculture department currently advise for them? Give me the link for each claim and the date shown on the page. If the only source is a seed company's blog, say so. Then the two or three most likely causes, how to tell them apart in my own field by tomorrow morning, and what I can do this week without buying anything. Last: the questions for my Krishi Vigyan Kendra, and whether there is a published advisory helpline for Kolar district - with the source.
Then loop it
- One of your links is five years old. Replace it with the most recent advisory you can find for this season, and give me the date on the page.
- Now put the questions for the agriculture officer into Kannada, short enough to read out on the phone, and export the whole answer to a Google Doc so I can show it at the KVK.
Why it worksThe photo shows the problem, but the words supply what the camera cannot - place, season, spread - and ask the AI to admit when it cannot see enough.
Instead of
Can you summarise this PDF for me?
Gemini version
[Attach the 40-page circular, or open it from Drive if it is already there.] Use only this circular. Do not search the web, and do not draw on what you know about pension rules in general - if the circular does not say it, it does not belong in the answer. Point yourself at Section 4 and the annexures. Check the remainder only for anything that amends Section 4, and name the page if you find it. Give me three things, in this order, with the page number after every single point: 1. The steps an employee follows, in sequence. 2. The documents needed, marked against the step they belong to. 3. Every deadline and time limit in the whole circular. Where the circular is unclear or silent, write “Not stated in the circular”. Where two parts disagree, name both pages. Then, separately and only after all of that: search for any later circular, memorandum or notification that supersedes this one, and give me the link with its date. Keep that part clearly fenced off from the part that came out of the document.
Then loop it
- Quote the paragraph on page 18 about the form, exactly as printed. No searching in this answer - only the document.
- Build the steps into a one-page checklist table - step, document, deadline, page - then export it to Google Docs so I can print copies for the employees at my desk.
Why it works"Using only the attached PDF" anchors the AI to the document, and page numbers let Fatima verify every answer herself.
Instead of
(spoken) Bill… why so much?
Gemini version
(Open Gemini Live. Photograph the bill first, with a thumb over the consumer number and the address, and this month's bill beside last month's if you can.) Spoken: “I am showing you my electricity bill. Talk to me in simple Tamil, and go slowly - I am 68 and I read very little. Tell me just two things to begin with: how much to pay, and the last date to pay. Say the amount twice.” (Then, still speaking:) “Now compare the units in this bill with last month's, and tell me in two or three short sentences why it has gone up. Do not read out the codes and the small print - only what I must know.” “Then two easy ways to use less current in the house, for one person living alone. Nothing that needs buying.” “If any figure is unclear in the photo, say which one and I will hold the phone closer.” (When it has finished:) “Now write all of that into a Google Doc in large print, so my daughter can read it when she comes.”
Then loop it
- (Spoken) Say the last date once more, slowly, and then tell me which day of the week that falls on.
- (Spoken) Is there a late fee or extra charge on this bill? Say yes or no first, then explain in one sentence. Do not search the internet - read it off my bill.
Why it worksVoice removes the barrier of typing and reading, but the prompt still states the aim, the language and the format - and personal details are hidden first.
Loop it
When you share a photo or file, the first answer tells you what the AI actually saw. Check that first: did it read the right page, the right column, the right figure? If it misread something, correct it - "The amount in row 7 is ₹1,250, not ₹125" - and ask it to redo the answer. If a photo was blurred, send a clearer one or a close-up. Then narrow the focus, ask for page numbers or quotes to verify, and turn the result into something you can reuse: a checklist, a table or a spoken summary.