Reflexions, the University of Liège website that makes knowledge accessible
Digital pathology is opening up new possibilities in modern medicine, and the era of optical microscopy may be coming to an end. Following joint efforts by computer experts and pathologists, new tools allow microscope slides to be digitised, annotated and classified using easily accessible platforms. With Project Cytomine, researchers at the University of Liège have stayed in the forefront of the development of biomedical tools. The advantages of the application involved are many. Slide information can be stored and shared, and displayed for teaching purposes. Above all, the programme performs part of the tedious work of analyzing biological cells and tissues, which is the daily grind of the pathologist. This software has possibilities that go beyond the university, and its continued development will be guaranteed through the launch of a spin-off in late 2014.
Since the 19th century, the procedure for observing and analyzing biological cells or tissues has not changed at all. A sample is placed on a glass slide, which can then be placed under the microscope and examined by a pathologist. With the advent of digital pathology, this procedure has begun to be superseded. This constitutes a minor revolution in the world of medicine. Various innovations, including the Cytomine software developed by researchers at the Institut Montefiore and the GIGA Centre at the University of Liège, make it possible to digitize, classify and share the information contained on slides with high image resolution and analytic clarity that is equal or superior to the clarity of optical microscopes. “The final result is a little like the technology of Google Maps,” said Cytomine project scientific coordinator Raphaël Marée. “Except instead of geographical maps, [the programme] displays biological samples that represent tens of thousands of cells or tissues”.
Project Cytomine was begun in 2010. It is an Internet platform that can store and classify digitized microscope slides, thus offering a number of new possibilities for cytology and histology, among other things. In the age of digitization, the procedure is almost disturbingly simple. Platforms and data banks are everywhere on the Internet. In the biomedical realm, the practical aspect of the project is emphasized. “Such technology for analyzing images didn’t exist 15 years ago,” explains Benjamin Stévens, computer expert at the University of Liège and coordinator for the spinoff launch. “Even Google Maps had not been developed yet. The costs of digitization were prohibitive, and they’re only now coming down. In addition, the speed at which images were processed was slow; it took more than 20 minutes to digitize one image. Today it only takes 2 minutes. The quality of the digitized image back then was not equal to the image provided by a precision microscope, or else the size of the image was too large for the memory capacity we had then, and Internet connections were not as fast, and experts were reluctant to make the change. That was understandable, they were responsible for diagnoses they made, and they trusted the equipment they were used to using. They had to be convinced that the tool we were developing would really help them work”.
But the new field of bioinformatics developed rapidly and soon won respect. In 2005 Raphaël Marée designed a generic model for the automatic recognition and classification of images as part of his doctoral research. There was nothing specifically medical about this development; the method could work with any images, and it was designed to classify them in terms of what they represented. This algorithm was much imitated. The technology improved around it, and when Raphaël Marée began to work with Benjamin Stévens, the programme was aimed at the biomedical field.
In 2008 their research team was contacted by an American company working on products that would assist in the detection of cervical cancer. The company was working on automated systems for the preparation of samples. That is, they produced slides and digitized them at a very high level of resolution (as high as 100,000 x 100,000 pixels). These images could be viewed on a screen instead of through a microscope. But the samples might contain hundreds of thousands of healthy cells, with perhaps only a few betraying the presence of a (pre-) cancerous condition. At this point, digitization has not yet made the pathologist’s task easier. If the researcher wants to detect tumorous cells, they have to be individually identified. “What the company wanted us to do was to develop an algorithm for detecting abnormal cells,” recalls Raphaël Marée. “An application that could find a needle in a haystack.
The second step involved figuring out how to get the programme to identify the things experts would be looking for in the images. In order to accomplish this, the researchers had to put the programme through a learning phase. “At first the programme has no way of classifying or segmenting images. So we asked the experts to show us cells representing the different categories they wanted to distinguish. Healthy cells, for example, or tumour cells. Basically we got them to repeat what they were used to doing with every cell in every sample”. Once the experts had “instructed” the programme, it was able through statistical inference and on the basis of recognition of colorimetric variations in pixels to classify cells rapidly, and in an exhaustive manner.
“So, this is a tool that with the help of experts can automatize tasks,” according to Benjamin Stévens. “At some point the experts, by annotating the images, shape the algorithm in a circular relationship. According to the first annotations the algorithm offers an initial prediction that the experts may correct. The algorithm remembers the corrections, recalibrates its values, becomes more exact, and eventually the model becomes robust”. The classification that is suggested automatically to the expert is thus at least as accurate as the manual activity of classification, and as exhaustive, because all the pixels in the image are taken into account. The algorithm can develop into a routine calculation, and the amount of time saved is then quite considerable. “The quantity of information that is sent to doctors is dramatically reduced,” Stévens said. “A doctor no longer has to hunt through thousands of cells, there are just a few hundred. The information is concentrated, made more reliable, and it makes diagnosis faster to help doctors. There’s no thought of replacing the diagnostic ability of a doctor, or the doctor’s expertise.” Another benefit: since the algorithm is for general use, and because it learns its task from experts, it is assumed that it can classify any kind of image. It only has to be given enough examples, and it performs the task automatically. Research into image analysis that is going on now is studying the validity of these algorithms with a wide variety of kinds of images.
That which functioned properly in a single programme of research on a specific cancer was assumed to work as well for an entire range of pathologies, whether this had to do with cytology or histology. This early collaboration gave the team the idea of developing Cytomine as a generic software platform for the analysis of histological and cytological images. With the help of Loïc Rollus, informatics specialist at the University of Liège, Cytomine would now take on a whole range of “intelligent” algorithms, as well as tools and interfaces that would be required before and after using the programme. “In addition, we gradually realized that the simple fact of being able to store and share these images filled a primary need, which allowed us to produce several different versions of the programme,” Stévens said. “But that was not out initial aim. We just wanted to create a tool that would make analyzing images easier”.
Over and above its ability to detect cell types, the programme was useful because of its ability to store and share images. From 2010, with the help of Professor Didier Cataldo, co-director of the laboratory of Tumor & Development Biology at the University of Liège, the Cytomine platform was in the process of development at GIGA. Its early successes allowed it to go from 5 users then to over 100 today, coming from the CHU and from the university at large. “We have opened up a similar service for students in histology,” Stévens said. “There are more medical students every year, and the faculty has to confront new problems of organization, especially where laboratory practice is concerned. Ideally, we need one microscope for each student, which is a great expense. Then, every student needs a slide for every exercise. We can’t repeat the same samples, and some are not as good as others. Getting good quality slides can quickly become a matter of difficulty. In addition, with the microscope the professor does not see what the student sees. Sometimes students miss questions because they did not observe closely enough. For so many students, microscopes are not very practical.”
Cytomine aims at offering obvious solutions to these problems. The lab is no longer obliged to own hundreds of microscopes, and a single good quality slide can be digitized and studied by all students at the same time, while the programme tracks their progress in observation. In an effort to stimulate interest, some teachers annotated certain parts of certain images in order to generate exam questions or student exercises. They present samples along with questions aimed at finding out if students have observed what they needed to observe. It would be possible to make a game out of certain searches, so that at every step students are alerted (with flags or other signals) regarding what they are supposed to be looking for. The programme thus has interactive pedagogical implications.
Today, over 1,000 student and faculty accounts are registered to use the programme. The team will participate along with the faculty of the medical school and with IFRES in the Histoweb project, an extension of Cytomine for teachers and professors, which will look for novel teaching methodologies, including online exercises intended to be carried, outside of class. “We will see when each student logs on. We will track them with a tool (with their permission) that will record some information about the way they prepare for their exams. We will be able to correlate these observations with eventual test scores as a means of improving students’ study skills”.
Outside of research and teaching, Cytomine could be valuable in a crucial setting, the clinical one: “This is an area where the replacement of microscopes is a controversial subject all by itself,” the researchers say. “Pathologists have to accept the technology. The diagnosis they produce based on an image has to be as good or better than the diagnosis based on something actually seen through a microscope.” There are three areas of potential difference. First, the possibility that digital compression of the image has altered it; next, the resistance of human experts to this technological change; finally, the eyes have it, because the field of vision presented on a flat screen is always smaller than the field of vision the eyes see through the microscope, including everything the eyes see, so that one can see right down into cells.
The mechanism of change is engaged, however. Taking into account only those who teach, the next generation of pathologists will have trained on flat screens. The leading microscope makers have accepted this change, and are now offering slide scanners at lower prices (a high speed scanner still costs about 200,000 Euros). At this moment, the CHU of Liège is asking this technology in specific kinds of cases. “For example,” according to Stévens, “when a surgeon operates, sometimes he wants to take a sample from a patient and have it analyzed quickly by a pathologist before going on. In a network of hospitals like the CHU of Liège a pathologist is not available at all hours, or in the right location. So an emergency call goes out, and the pathologist may spend quite some time driving for just a few minutes work with a microscope. With this technology the doctor can take the sample, and a technician can immediately digitize it. The pathologist gets the image as an e-mail and is able to answer the surgeon’s questions in a few minutes, after which the operation can continue.
Cytomine has been a big hit with the medical faculty of the ULg, and is ready to make a wider circle of friends. There was already interest. “In the pharmaceutical sector, the Breast International Group (BIG) is interested in our platform,” says Stévens. “Their research is aimed at the most severe form of breast cancer, a type against which science has not made progress in the last 20 years. So the BIG company launched Aurora, an international project, for the purpose of gathering, classifying and analyzing 13,000 slides from 1,300 European patients. We will let them use our platform for handling imagery.” The struggle against cancer has to be carried on at the international level. It is impossible to put together a reliable data base if all the data comes from one country such as Belgium. There are not enough patients here. Means of communication, central facilities for storing and sharing images and easy access to the system – these are the priorities.
Another possibility involves the use of a bio-bank, a collection of tumour tissues, with samples from different studies divided into groups and classified after being digitized. “We are in contact with other large research centres. These centres of cancer research allow many researchers to come work for them for two or three years. Each one takes hundreds of samples for their work, and these end up in file drawers, and there is no traceability for the slides. The centres want to end that waste. They systematically digitize all samples taken, and they will need a programme like ours to keep track of their activities and to provide content for their bio-bank. The bio-bank will be at the service of future researchers, especially by reducing the cost of research, and it will reduce the need for producing so many slides”.
Taking note of these needs, shared by many, the project leaders have decided to bring the project out of its university campus cocoon. The programme is now robust enough to survive being adapted to commercial requirements. Raphaël Marée will remain at the ULg, developing algorithms for image analysis for the platform, and he will work on bringing out the pedagogical potential of the programme. Benjamin Stévens will take the project is yet another direction: “We have a First Spin-off Project that should appear at the end of 2014. The purpose will be to get closer to the client, from installation to the specialization of the programme, so everyone’s expectations will be realized. We will offer storage space and other specific developments depending on the need… Over and above the relationship with the client, this Spin-off will represent a high-quality brand”.
Success seems within reach for Cytomine, which has only begun to be applied outside the biomedical realm. Groups from various areas are asking for access to the platform, using it for storage and classification. "There are dentists, geologists, and veterinarians. One could imagine that the programme could also be useful in astronomy, botany, and geospatial analysis. Derived from digital cartography and eventually returning to it, that’s a worthwhile circle, is it not?” In the meantime, Cytomine continues to win acceptance in the biomedical area, and has still other opportunities in view. “Digital pathology is still a luxury,” Stévens concludes before presenting a final example of the possibilities of the spin-off. “Scanners are very expensive, the technology is not always available… We want to be able to offer digital pathology at low cost. To use the platform, all you have to do is create an account. As for digitization, the platform’s clients will have reduced need for investing in a scanner for spot duty. All they will have to do is deliver their samples, which we can scan in the tumour biology and localized development laboratory at the CHU in Liège (we are able to scan 1000 slides a day), and they will receive links so they can view the digitized images online."
© Université de Liège - http://reflexions.ulg.ac.be/cms/c_372880/en/digital-technology-may-make-microscopes-obsolete?printView=true - May 26, 2017