In this tutorial we'll show how Mathpix OCR can be used to instantly transpose math and science materials in a way that can be read by Blind and Visually impaired students. In the next section we'll briefly describe technologies used by blind people to access to electronic information. Then we'll present the EDICO scientific editor and the Mathpix software. In the section 4, we'll discuss the process a teacher use to convert a math assignment from a PDF into a format that can be read by a blind student. Finally in section 5 we'll demonstrate how the student can read and solve the assignment.

What we want to point out here is that we're not just discussing a way to make math and science texts readable by students who are blind or visually impaired. We're discussing a viable method that, thanks to the powerful Mathpix and EDICO software working together, let blind people read, write and solve math and scientific problems as their sighted schoolmate are able to do.

A blind person who wants to access to electronic information interacts with the computer by using a software called screen reader.

Screen reader software are available for all major operating systems including Microsoft Windows, Mac OS X and Linux.

Screen reader are specifically designed to be used by blind people, that interact mainly by using the keyboard. By the use of different key combination (called keyboard shortcuts), a blind person can move around the screen, interacts with GUI control and access to the installed application including web browser, email clients, word processors etc.

For more information about how a screen reader works you can refer to The ability net webpage on An introduction to screen readers.

A screen reader let the users decide how to output information. It can use speech, by using a technology called TTS (Text To Speech). A speech synthesizer reads aloud the content of the screen and the user can decide to read one character at a time, by word, by sentence etc. This is not so convenient to explore scientific formula, because the user needs to remember the whole formula in its mind to get a complete overview of it.

In addition to speech synthesizer users can access information by using a Refreshable Braille Display. They provide a tactile access to a text line by representing it in braille.

The braille system of raised dots, a way to represents information that can be read with the fingers by people who are blind or who have low vision. Braille offers a linea representation of information. As reported by AFB, Braille symbols are formed within units of space known as braille cells. A full braille cell consists of six raised dots arranged in two parallel rows each having three dots. The dot positions are identified by numbers from one through six. Sixty-four combinations are possible using one or more of these six dots. A single cell can be used to represent an alphabet letter, number, punctuation mark, or even a whole word. For more information about the Braille System please visit the AFB page related to Braille.

Electronic braille is formed by braille cells of 8 dots. In this way all possible characters in ASCII can be represented, since all possible combinations are 2⁸ = 256 patterns which can be use to represents all differents 256 character.

As reported by AFB, braille display can show up to 80 characters from the screen and is refreshable—that is, it changes continuously as the user moves the cursor around on the screen, using either the command keys, cursor routing keys, or Windows and screen reader commands. The braille display sits on the user’s desk, often underneath the computer keyboard.

Braille cells are dynamically refreshed by push up and down plastic pins, driven by a specifically designed software that interacts with a screen reader. For a more advanced description on how a braille display works Refreshable Braille Now and in the Years Ahead article is a good starting point. In Internet are available a lot of videos that show how a user can access information using Refreshable braille display.

The most relevant issue in access of math and science text, using both speech and braille display, is that in many times it is not represented in a linear way (latex, ascii-math, markdown etc). Instead it is represented only in a graphical representation. Screen reader are unable to offer access to information that is not structured. In the case of Math, a screen reader can offer access to MathML, because it is structured, but it can't offer access to a PDF file.

There are a lots of tools that can be used by authors to produce accessible PDFs containing scientific formulas. Actually, they are not largely adopted, because only few people knows about this problem, and, by this way, blind and visually impaired are condemned to remain distant from a full access of scientific information.

A more complete dissertation on this can be found on Access by Blind Students and Professionals to Mainstream Math and Science, by John A. Gardner.

Mathpix Snipping Tool is an AI-Empowered OCR that can be easily used by teacher to "take a snapshot" of a math formula and convert it in MathML. Using this tool, the Teacher don't have to manually transcribe examples, tests and homework to be specifically accessible to blind students. She simply take a snapshots of the formula, a process that requires just few seconds, and the blind student can instantly read it using his favorite screen reader rendered both in braille and in speech.

By using Mathpix you can export formula in MathML, so that it can be accessed using advanced technologies contained in recent screen readers, you can export it in LaTex, so that blind students who knows this linear language can easily read and understand it, or you can export it in ASCII math, an easy way in which math formula can be represented.

As we have discussed on the introduction of this tutorial, this is only a part of the problem. By this, the blind student can read the formula. But she can not interact with it. She can read the equation, but the objective is that she learn how to solve it. This is where EDICO comes into play.

EDICO is a Scientific Editor specifically designed for the blind. It is developed and maintained by the ONCE, the Spanish Organization for the Blind and the UCM University. You can find more information and download it for free from the EDICO page on the ONCE Website.

The interface of EDICO has been designed to let blind and sighted people working togeter on scientific texts. It is well integrated with the two major screen reader for the Microsoft Windows OS, so that it can provide information both in braille and in speech.

The screen is horizontally splitted in three parts. In the first part we find the linear representation of the document. It is in a format, specifically designed to be easily readable by blind people using a braille display. In the second area, we'll see a "real" representation of the formula, where a sighted person can read math and chemistry as she's used to. At the bottom there is a visual representation of dot patterns that the blind student read on his braille display. By this, a teacher can help him in understanding the braille and can teach him how to approach to the math using this special tool.

The user can move around the formula, by using arrow keys. When it moves, the speech syntesizer reads the character on the cursor position. The student can also write symbols and letters using keyboard. Keyboard shortcuts are provided for open/close construction like fraction, braces, parenthesis, sum etc. Specific tools have been designed to help the blind student in understanding how the formula is structured and to make the formula exploration easy to do.

The process is as fast as simple. First of all be sure to have both the MathPix and the EDICO software installed on the computer where you have to perform the transposition.

Then we select in the MathPix editor to export the formula in LaTeX format:

Then open the PDF containing the math exercise or scan it using a flatbed scanner.

For instance, we have opened this PDF freely available online from the Italian Zanichelli Website: https://online.scuola.zanichelli.it/bergamini-files/Biennio/Recupero/bergamini_monomi_R3_5VB.pdf

We then press the shortcut key Control-Alt-1 and we select the area of the equation we want to transpose using the mouse.

Finally we open EDICO and select the "Paste from LaTeX" option.

Then we fix some small semantic errors (like power represented as superscript) from this:

And the exercise is ready to be solved by the blind student!

After this, a blind student can read the expression using her braille display. First of all she copies the whole exercise without the exercise title and explores it with the EDICO powerful Structure View tool.

She is able to identify the three bloks of the formula and to solve each single small block:

and then to solve it

And to put all together for the final result:

Finally the completed homework can be exported as a PDF file or printed to be checked by teacher.

In conclusion we've show how an AI Empowered OCR like MathPix Snipping Tool can work together with a specifically designed Scientific Editor to let blind students access math and scientific materials and to solve complex math assignments.