Pixels

1. Moving from 2D (two-dimensional) to 3D (three-dimensional), we move from pixels (picture elements) to voxels (volume elements). Pixels are more commonly known than voxels. Voxels are just the 3D counterpart of pixels, which are 2D. A 2D image is composed of many tiny squares. A 3D volume of images is composed of many tiny cubes. In 2D, we have one dimension for width, another dimension for height. In 3D, we get an additional measurement: depth.
2. See the lungs, the heart and the remaining anatomy? This is a cross-sectional cut of someone's upper torso. It looks kind of different, because medical images are usually not represented this way. But the different organs are completely recognisable. Here, each pixel houses a character. Some pixels house the same character, others don't — it is from the similarities and differences in the characters housed that we make out the anatomy.
3. This is the same image represented in a more conventional way — each pixel is filled with a colour — some pixels share the same colour, others don't — it is from the similarities and differences in colour that we make out the anatomy.
4. Back to the pixels housing characters. This image is in fact saved simply in text format, with file extension '.txt'. Completely rudimentary. Formats like jpg, gif, png etc would have been taken larger diskspace. We can in fact count how much diskspace we need for this text file. We have 86 rows of pixels. Each row consists of 147 columns of pixels, plus an invisible 'newline' or 'carriage return' marker at the end. Without these invisible newline markers, our image would have been a single row of 86 x 147 columns of pixels and we won't be able to make out any anatomy. No, that's not what we want. So we do need the invisible newline markers, which take up their small share of diskspace. The diskspace of the text file is therefore 86 x (147+1) = 12728 bytes.
5. Here, again I have used the pixels-housing-characters format to show you an image. Guess what I'm trying to show you. Not easy. Because, this format can display images up to a certain point only. We need to resort to colours filling up pixels.
6. Voila, let there be colours. Here's what I was trying to show you. Darling Pablo listening to me.
7. The audience might feel attracted to any possible tips on how to decode CT, MRI, X-ray, ultrasound, PET images. But this talk offers nothing whatsoever of that sort; one would be really disappointed. Interpreting and diagnosing medical images are part of a radiologist's remit. I'm no radiologist. I'm a physicist. My role is to ensure that features on images faithfully represent the inside of the patient's body. The radiographer's role is to position the patient in the best possible way, execute the best possible protocol and present the images to the radiologist. Together the radiologist, the physicist and the radiographer work hand in hand, striving towards the best diagnosis — either true positives or true negatives, never false positives or false negatives.
8. Apart from X-ray images, most modern-day medical images are in 3D. With a set of 3D data, we may make cross-sectional slices whichever way we like. Transverse, sagittal and coronal are common cuts. We can even make oblique cuts. Just an example — don't bother to try imagining and getting the example right. Most of us do not think in 3D. Some people are able to; that gift is a truly great asset.
9. The pixel value (or pixel intensity) may represent different quantities, depending on the imaging modality. Pixel values on photographs are a measure of visible light reflected by objects. Pixel values of CT (computed tomography) images are the Hounsfield Units (HU, or CT numbers) which measure linear attenuation of photons by the objects. Pixel values on PET (positron emission tomography) images measure radiotracer (¹⁸F, for example) uptake.
10. See how the photo becomes 'not so nice' with the addition of noise and the degradation of resolution. Noise makes the photo grainy. Poor resolution blurs the photo.
11. Let us move from photos to medical images. See the adverse effects of adding noise, degrading contrast, and degrading resolution from 105x147 (105 rows of 147 columns o pixels) to 53x74 (53 rows of 74 columns of pixels).
12. High resolution alone does not promise good image or display quality. Contrast, noise and resolution are the three quantities for gauging image quality. The sharper the contrast (less blur), the higher the image quality. The lower the noise, the higher the image quality. The smaller the pixel size, the higher the resolution and therefore the higher the image quality. Sharp contrast alone, however, does not imply good image quality — we must still check the noise level and the resolution. By the same token, low noise, or high resolution, alone cannot make a good image. For an image to be of high quality, we need a combination of all three factors: sharp contrast and low noise and high resolution. The implications of contrast, noise and resolution of medical images are not whether the pictures look nice or not. They may cause artefacts or misrepresentations suggesting features which are in fact absent in the body of the person scanned. They may also cause important features not to be visible. This would lead to wrong diagnosis.
13. For pixel graphics, magnifying the image without increasing the resolution would degrade display. We would see step artefacts where smooth surfaces become stepped or sawtoothed. Vector graphics, on the other hand, would not have this problem when magnified.
14. Here's an example. Zooming in pixel graphics will degrade the image. Zooming in vector graphics won't.
15. By the way, here is the making of the pair of hands which occasionally appears on this website (dirty hands @ work, the lecture about voxels and the 5-finger prayer): both hands are really mine. My right hand, which had a fracture when I was thrown off a mule like a projectile back in September 2010. We were enjoying a beautiful walk on a wonderful day when she broke into an unplanned gallop. I took the X-ray images and superimposed my bones over my favourite textile using GIMP — a fantastic open-source alternative to Photoshop.
16. Given an image, can we reduce the noise? Yes, denoising is a commonly available tool. We can denoise the image, usually at the expense of resolution. Most denoising algorithms inevitably smoothen. Given an image, can we improve the contrast? Yes, through edge detection and sharpening. Where pixel values jump, we can exaggerate the difference and make the gradient even higher. Given an image, can we increase the resolution? For pixel graphics, no! Even if we upsample, say, from 256×256 to 512×512, we won't be able to recover information which was never there in the original 256×256 image.
17. Partial volume artefact: we cannot have any details within a pixel. A pixel can only take one pixel value, or one colour. It cannot be partly this colour, partly another colour. The bigger the size of the pixel, the lower the resolution, partial volume artefact becomes more pronounced.
18. All four images on this slide have a common resolution. Low noise alone cannot give us a good image unless contrast is good (high). High contrast alone cannot give us a good image unless noise is low.
19. All four images on this slide have a common contrast. Low noise alone cannot give us a good image unless resolution is good (high). High resolution alone cannot give us a good image unless noise is low.
20. In an imaging clinic, nurses are often needed to administer cannulation on the patient. Contrast media (or contrast agents) are either injected or ingested into the body. These foreign compounds accumulate in specific tissues only, highlighting specific areas to help radiologists see.
21. Modern displays often allow the user to choose from a variety of palettes, or colour maps. Grayscales might appear least exciting but they remain the colour map of choice in many medical diagnoses. Note the difference between grayscales and black-and-white. The two are far from identical!
22. Here is exactly the same image data displayed with a different colour map.
23. Here is yet another colour map displaying exactly the same data.
24. Here is yet another colour map. Colourful, but bad, because the colour bar recycles the same set of colours on different pixel values. Colour representation of pixel values is no longer unique. We can no longer infer the pixel value from the pixel colour.
25. But there is one thing good in this display, in that we can delineate white matter from gray matter better. It is so happen that the pixel value for gray matter is coloured green here whereas the pixel value for white matter is coloured purple, and green and purple look more strikingly different compared to the colours on the image shown two slides earlier. The morale here is not suggesting that we should use this colour map, but that different colour maps can be good for different purposes.
26. Here is the colourmap usually referred to as 'hot'. Note that the pixel value is uniquely mapped to colours without any recycling or repeat. Incremental colour change is monotonic. This is my preferred colourmap, especially for publications, because it displays correct (without repeats) even when printed in grayscale, without colours. We cannot assume that all readers print in colours, or have colour printers.
27. Here, I invert the colourmap. Left: pixel values increase from dark to bright. Right: pixel values increase from bright to dark. Both are equally valid. Both display exactly the same data, consistently. Although there are clinical conventions such as:
    • bones are bright and lungs are dark on X-ray and CT images;
    • breast tissues are bright against a dark background on mammograms;
    • points of high uptake are dark against a bright background on PET images;
    • fat is bright and CSF is dark on T1 MRI;
    • CSF is brighter than fat on T2 MRI,
    so much so that inverting the colourbar may appear scandalous, physicists should always think in terms of pixel values rather than colours.
28. Colours are really secondary on images. We should instead worry about misrepresentation of features (artefacts). Features which are not in the patient's body should not appear on the scan. Features present in the patient's body should be visible to the radiologist. It is the physicist's duty to ensure so.
29. Windowing changes the appearance of an image. Different windowing can expose and hide certain features. Limiting the window so that the window is smaller than the full range of pixel values will expose more details within the window, but will lose details outside the window. Here, window refers to the range of displayed pixel values, not the geometric field of view.
30. Setting the window wider than the full range of pixel values will reduce contrast by not making full use of the full range of colours.
31. We lose some features and gain other features when we zoom into a narrow window.
32. Inherent contrast is limited by the detection system. We can't change the inherent contrast by windowing but we can tune the display to taste by windowing.
33. The two gentlemen starred in this slideshow...
34. Monsieur Pablo, the darling mule who listened like a horse. Who says donkeys are dumb? I took the picture, myself on his back.
    Mr Joseph Paul Jernigan, who kindly consented to the prison chaplain to donate his body for scientific research or medical use. The Visible Human Project sliced up his corpse at 1 mm intervals. I believe he did not envisage such a global fame prior to execution by lethal injection. He posed himself as my virtual patient and received countless radiotherapy shoots in my research. I'm against execution and I do not believe in punishing except as deterrent; I count Mr Jernigan one among the smiling species. I choose not to express this stand of mine by shying away from this precious data collected from a human body.
35. Image quality quantified. Basic: spatial resolution, contrast, noise. Integrating: MTF (modulation transfer function) and SNR (signal-to-noise ratio). Unifying: DQE (detection quantum efficiency).
36. Physicists should be able to recognise different QA phantoms, even without seeing the model or brand before. Phantoms with line pairs test spatial resolution. Phantoms with areas of varying sizes against a background of a different colour check contrast. Phantoms with areas of the same colour check uniformity.
37. The origin of noise ...
38. Scatter causes noise. So what's wrong with scatter? Compton scattering, for example, emits the photon at an angle away from the original. When detected, this photon would suggest an origin which is not true, causing noise. Photoelectric absorption, on the other hand, do not emit scatter photons.
39. Physicists needs to be able to estimate the file size of medical images, which is the total number of voxels multiplied bytes per pixel, plus the size of the metadata, which is present in all DICOM (and some NIfTI) files.
40. This is how the total number of voxels is calculated.
41. This is the pitch.
42. Bytes-per-pixel depends on the colour depth. Deeper depths require more bytes per pixel and allow more shades on our palette.
43. 1 byte = 8 bits. One byte allows 256 colour depths, because the number of possible numbers in the range 0 to 1111 1111 is 256. This is the reason why we always see 256 and 512 if not 1024.
44. The metadata.

Let us start with a quick comparison between digital photos (or pictures) and medical images:

• Colours are more important on photos than on medical images.
• Pixel values are more important on medical images than on photos.
• Local features are more important on medical images than on photos.
• Digital photos come in 2D (on a sheet of paper, or a flat screen), medical images come in 3D (a stack of 2D images).

Colours and pixel values

Pixel values are less important on photos and pictures. In fact the very same photo may be represented using completely different pixel values e.g. RGB and HSL. For medical images, however, I don't care whether a pixel shows black or white, red, green or blue, purple or teal — so long as there is a colour-bar showing me the pixel values corresponding to the colour scale. The meaning of medical images lies in the pixel value, not the colour. For example:

• X-ray and CT images have high pixel values for bones, low pixel values for lungs. Increasing pixel values are typically displayed with a colour bar stretching from black — through a range of gray-scales — all the way to white.
• PET images have high pixel values for high counts, low pixel values for low counts. High counts are typically displayed as dark spots (high counts) against a white background (zero count). This colour scale is opposite of that conventional for CT. There are indeed conventions hospital staff get very accustomed to, and might find alternate colour-maps scandalous.

While gray-scales may invite fond nostalgia of black-and-white televisions, they should never be under-estimated. This colour-map is still widely recognised as the one offering the best medical diagnostic value. Note that gray-scales are not black-and-white. We can have thousands of shades of gray. Black-and-white is limited to just two colours: black and white.

Local features

Although one might feel conscious and tries to remove wrinkles and a pimple or two from photos, local features are far more important on medical images than photos. A local feature makes a big difference to a person's life, casting the diagnosis into one of the four types: false negative, true negative, false positive or true positive. The latter pair may open up a whole episode of medical interventions.