from apprentice to master

When I grew up on the western edge of Lugano, life in the old country was not very mobile and people tended to spend their life at audible distance from the church bells they heard at birth. The most exotic person you could come across was that wiry old man in a tattered white suit hiking in the woods of Collina d'Oro that was Hermann Hesse, often carrying a drafting book and a box of water colors, totally absorbed in his thoughts.

With this, when Ravi Shankar came to town for a concert sponsored by Migros, it was like a little prince coming from an other planet. Indeed, he and his daughter were wearing traditional Indian clothing, something that had never been seen before on the streets of Lugano. I had the chance to spend a full day and an evening chatting with him, while conducting an interview for the youth radio and helping setting up the recording equipment for the concert.

I was amazed by the difference in personality between him and his daughter. The latter was distant and exotic, but Ravi Shankar was immediate and approachable. It was easy to talk casually with him. I had never seen a sitar before and started to talk with him about the complexity of the instrument. Since he was classically educated, I asked him how long it had taken him to become a master, compared to the piano or the violin. He countered he was not a master at all. When I noted his total command of the sitar and how he appeared to be one with it—even citing Jimi Hendrix and his electric guitar—he rebutted that he was just an apprentice. He continued that the sitar is an instrument that takes a whole life to learn to play, and it is only after reincarnation that one can play it as a master. He pointed out, that when performing on a complex instrument, total mental concentration was necessary and mastery of the sitar takes two lifetimes.

During the concert, I was at the left side of the first row and soon noticed that he was not only in constant musical contact with the other players, but he was also conversing with me through his eyes. Indeed, his protocol was to fixate a member in the audience he thought had an interesting posture, then fixate me, return to the first person to guide my gaze, and when I followed his gaze and looked at that person, he acknowledged with the hint of a smile. While Herman Hesse was totally absorbed in his thoughts, Ravi Shakar was involved concomitantly in three conversations—with the sitar, the orchestra, and the audience—meaning he had full awareness of his surrounding while he was also fully absorbed in the music.

Yesterday Ravi Shankar reached the end of his apprenticeship and today—12/12/12—he is a master sitar player.

Extending the printer gamut upwards

Full color printing started with cyan, magenta, and yellow. Then black was added to extend the gamut down in the shadows. Later spot colors were added to make the gamut wider (Hexachrome, Indichrome, etc.). Now Romain Rossier & Roger David Hersch are adding light fluorescent magenta and yellow to extend the gamut up in the light colors. They are presenting their work at the CIC 20^th Conference in Hollywood in the Friday afternoon session on Printing chaired by Jan Allebach. Of course, the slides are limited by the projector's gamut, so you need to be there and look at the actual prints.

GPU-accelerated Path Rendering

Last May I wrote about a major breakthrough in path rendering on the GPU by Mark Kilgard. I am happy to report that Mark—together with Jeff Bolz—has been hard at work on the rest necessary for a complete raster image processor (RIP). They have invented a new "Stencil, then Cover" (StC) algorithm in which the stencil step is explicitly decoupled from the subsequent cover step.

In the stencil step, a path's filled or stroked coverage is determined. In the cover step, the conservative geometry intended to test and reset the coverage determinations of the stencil step is rasterized, while shading color samples within the path. They have not only achieved fantastic acceleration, but also full completeness and correctness. Usually, the performance killer is the bottleneck between CPU and GPU, like when transparency is computed in the CPU. Kilgard and Bolz solve their revalidation bottleneck by using a configurable front-end processor in the GPU to transition quickly between the stencil step and the cover step.

For more information, see their paper at SIGGRAPH Asia, a pre-print of which is available at this link: http://developer.nvidia.com/game/gpu-accelerated-path-rendering.

Binarization

This year I have been working on document information retrieval, which is as far from color as you can imagine. Indeed, business documents are pretty dry binary black and white items, so that the first step—before even doing optical character recognition—is to binarize the document images so we can efficiently work with bitmaps. In the old days binarization was relatively easy, because almost any scanner illumination can easily be compensated when it is not uniform (see US patent 5,901,243).

Today binarization is much harder, because an increased number of documents is imaged with digital cameras, most often of the kind in smart phones. Much work went into extending existing binarization algorithms to text in pictorial images, alas with little success. It turns out that a completely different algorithmic approach is required, as was recently published in the paper

Yan Wang and Chuanjiang He, Binarization method based on evolution equation for document images produced by cameras, Journal of Electronic Imaging 21 (2012), no. 2, 023030

Here is the abstract:

We present an evolution equation-based binarization method for document images produced by cameras. Unlike the existing thresholding techniques, the idea behind our method is that a family of gradually binarized images is obtained by the solution of an evolution partial differential equation, starting with an original image. In our formulation, the evolution is controlled by a global force and a local force, both of which have opposite sign inside and outside the object of interests in the original image. A simple finite difference scheme with a significantly larger time step is used to solve the evolution equation numerically; the desired binarization is typically obtained after only one or two iterations. Experimental results on 122 camera document images show that our method yields good visual quality and OCR performance.

Blue nights

I usually sleep with open curtains. Fireflies aside, growing up, when I woke up in the middle of the night and looked up, I saw the Milky Way. It was quite a change when as a student I moved into a room in Zürich's Predigergasse that had a gaslight on the corner of the house, just outside my windows. When I woke up in the middle of the night, instead of a black firmament with the twinkling Milky Way, I experienced being bathed in a flickering red light. My insomniac nights changed from black to red.

Our house in Palo Alto features a street light smack in front. At night, monochromatic yellow sodium light shines through the large picture window and paints the house in a warm light, emphasizing the polenta-yellow walls and the golden white oak hardwood floors.

Until yesterday.

The City replaced the sodium lamp with an LED lamp and the floors are now patterned by cold blue reflections, which in turn create eerie light plays on the walls. Instead of averaging and blurring the material structures, the new light analyzes and emphasizes them.

It will take some time getting used to it.

By the way, in 1980 low pressure sodium street lighting became common in the Silicon Valley in support of the Lick Observatory on Mount Hamilton, to reduce light pollution. The Barron Park neighborhood in Palo is dark at night, because the inhabitants keep their porch lights off. Their neighborhood park is named after Cornelis Bol, a Stanford physicist and the inventor of the high-intensity mercury vapor lamp.

Avoidance of plagiarism

As the center of gravity for scientific research is moving from old locales with longstanding traditions to new geographies with forgotten traditions or which never had them, plagiarism has become a major issue. At times it feels like being in the Wild West.

Sometimes papers are pure copies of previous work. More often, authors have not yet developed a proper hygiene for citing related work. I am always surprised how often authors cite secondary references instead of primary references, a fact I tend to take as laziness and punish with a negative review.

In Volume 6, Issue 1 of the SPIE Journal of Nanophotonics, Editor-in-Chief Akhlesh Lakhtakia has written a very useful editorial on this topic. Follow this link: http://dx.doi.org/10.1117/1.JNP.6.069901.

Firefly

The firefly plays a special role in Japanese culture. It is called hotaru (蛍, ほたる) and we researchers know it from the expression 蛍雪 (けいせつ), or firefly-writing, which refers to diligence in studying (i.e., continue to study even in such poor light as offered by a firefly). For more wordly people, the firefly is the symbol of passionate love.

At least from the 24 April 1185 battle of Dan-no-ura, where the Genji under Minamoto no Yoshitsune, defeated the Heike (Taira), if not from earlier, it is believed that when soldiers are killed in battle, their souls are transformed into fireflies. Therefore, in Japan the view of hotaru is very sentimental and patriotic.

Today the life of scientists is more peaceful, as researchers are no longer killed like Goethe's Faustus when his grant was up or Giordano Bruno when he came up with the mathematical concept of infinity (see glad not to be on the stake), so we can be cheerful when we see fireflies.

I remember when we moved to Lugano, at the city's border, consisting mostly of untended fields. The place did not even have a name yet, it was just the far end of Besso, or Lugano 3, as the postal system prosaically called it with the introduction of zip codes. As kids we only had to run away from the apartment buildings for a few minutes to be in a completely dark environment devoid of any light pollution. The black sky was dotted with infinite stars, but in summer, towards Cortivallo and the lake of Muzzano, we were immersed in a cloud of fireflies. It was a magic experience.

Of course, today as color scientists we are more interested in the spectrum of the firefly. Entomology teaches us that males and females are anatomically different, with the latter having two lateral light sources and the former three adjacent light sources. This means that we have to measure the sexes independently. How can we achieve that?

In his recent paper in Atti della Fondazione Giorgio Ronchi, volume LXVII (2012), number 3, pages 455–458, Paolo Stefanini reports how he accomplished it.

The males normally cruise above the fields, while the females are hidden in the grass. When the females want to mate, the crawl to the apex of the grass leaves and wait. A males ready to mate flashes his light and a female flashes back, then they go at it. Therefore, Stafanini first measured the males, then he built a male decoy using LEDs. The decoy allowed him to beat the females out of the bushes, so he could measure them too.

La tienda de las curiosidades sobre el color

Gladstone famously asserted Homer—and in fact all Ancient Greeks—were color blind because Homer's works never mentioned a color. I do not know what Gladstone was reading in school, but I remember reading Homer's vivid descriptions of sunrises and sunsets, not to mention the descriptions of light at sea, for example when Odysseus tried to withstand the song of the sirens.

This demonstrates the need to teach the perception of color: if we cannot express it, it does not exist. Today, the world is much more colorful, so it has become an easier task. I remember when I was a kid and the world was much more monochromatic: cars were either black or white, and most clothes were grey. In elementary school, we boys had to wear a black smock, while the girls wore a white smock. Next door from our apartment building, Pasticceria Ceroni had the first TV in Bellinzona, and it was black and white. Later, when color TV came on the market, it was so expensive, that kids used to brag by exclaiming: "my dad makes more money than yours, we have a color TV." Meanwhile my family was so poor that even our rainbows were just in black and white…

Despite all the color stimulation everybody can enjoy today, it still must be sorted out and explained, otherwise kids cannot enjoy the rainbows in color. April 27, Mark Fairchild and Manuel Melgosa Latorre presented a new book edited by the Editorial Universidad de Granada (eug), in collaboration with the Parque de las Ciencias that makes a wonderful contribution toward this task: “La tienda de las curiosidades sobre el color.”

El libro “La tienda de las curiosidades sobre el color” da respuesta a 64 preguntas didácticas sobre el color: ¿Cuántos colores hay en el mundo?, ¿qué es el color?, ¿cuál es el mejor color para unas gafas de sol?, ¿por qué el cielo es azul?, ¿puede un perro ver el color lo mismo que yo?, ¿por qué tienen las flores distintos colores?, ¿por qué no podemos ver los colores de noche?, ¿por qué mis fotos no tienen los mismos colores que las escenas originales?

Estos son algunos de los interrogantes que el libro desvela. Las preguntas están ordenadas en 8 módulos (luz, objetos, ojos, etc.), y dentro de cada módulo hay 8 niveles de creciente complejidad, ya que el libro va dirigido tanto a estudiantes de primaria como a estudiantes universitarios, y también a cualquier persona interesada en la ciencia.

The book is also available in English: Explore Mysteries of Color — Discover Why is Color.

Colorful Language: The Results

It has been almost a year since we mentioned Eleanor Maclure's survey on Colorful Language. First of all, congratulations to Eleanor for a successful graduation! In fact, she has since completed the course and produced an illustrated report of the results of the survey.

The report is easier to look through on Eleanor's issuu profile, but she also has the PDF available to download from her blog. There are other parts to the project which are more visual explorations of color and language as well, she did a number different things for her MA because it is such a big area to study.

Enjoy!

war of currents

Power distribution is a key technology for increasing the efficiency of work, thus increasing life quality and improving the human condition. Early factories had extensive belt systems to transmit the mechanical force of a water wheel to the workstations in a plant. In the industrial revolution the water wheel was replaced with the more predictable and powerful steam machine, but the inefficient and inconvenient mechanical distribution through belts remained.

Color naming models at CHI'12

At CHI'12, the 2012 ACM annual conference on Human Factors in Computing Systems, Jeffrey Heer and Maureen Stone presented a paper on color naming models for color selection, image editing and palette design. It has a good section of the statistical tools for cleaning up a large crowdsourced corpus like the XKCD color naming experiment.

Link: http://doi.acm.org/10.1145/2207676.2208547

Abstract: Our ability to reliably name colors provides a link between visual perception and symbolic cognition. In this paper, we investigate how a statistical model of color naming can enable user interfaces to meaningfully mimic this link and support novel interactions. We present a method for constructing a probabilistic model of color naming from a large, unconstrained set of human color name judgments. We describe how the model can be used to map between colors and names and define metrics for color saliency (how reliably a color is named) and color name distance (the similarity between colors based on naming patterns). We then present a series of applications that demonstrate how color naming models can enhance graphical interfaces: a color dictionary & thesaurus, name-based pixel selection methods for image editing, and evaluation aids for color palette design.

First Ever Retinal Tissues From Stem Cells

A research team from the Riken Center for Developmental Biology and Sumitomo Chemical has produced stratified retinal tissues from human embryonic stem cells for the first time in history. The results have been published in the scientific journal Cell Stem. In 2011, the group had managed to form an optic cup from ES cells in mice. ES cells are capable of replicating into any type of cell. The method was applied in the latest research, in which about 9,000 human ES cells were cultured. An optic cup, an early stage of development of an eye, was created in about 25 days. In the 18th week, it had developed into a three-dimensional neural retina with photoreceptor and ganglion cells.

[Source: Felix Moesner, Science & Technology News from Japan, June 2012]

Switzerland Retains First-Place Position in Innovation Performance

Today, INSEAD, the leading international business school, and the World Intellectual Property Organization (WIPO) released the Global Innovation Index 2012 (GII): Stronger Innovation Linkages for Global Growth. For the second year running, Switzerland, Sweden, and Singapore lead in overall innovation.

"The GII is a timely reminder that policies to promote innovation are critical to the debate on spurring sustainable economic growth," WIPO Director General Francis Gurry said. "The downward pressure on investment in innovation exerted by the current crisis must be resisted. Otherwise we risk durable damage to countries' productive capacities. This is the time for forward-looking policies to lay the foundations for future prosperity."

The list of overall GII top 10 performers has changed little from last year. In parenthesis is the score (0–100)

1. Switzerland [68.24]
2. Sweden [64.77]
3. Singapore [63.47]
4. Finland [61.78]
5. United Kingdom [61.25]
6. Netherlands [60.55]
7. Denmark [59.93]
8. Hong Kong (China) [58.72]
9. Ireland [58.68]
10. United States of America [57.69]

Canada is the only country leaving the top 10 this year, mirroring weakening positions on all main GII innovation input and output pillars. The report shows that the U.S.A. continues to be an innovation leader but also cites relative shortfalls in areas such as education, human resources (tapping of global talent) and innovation (research, patenting, and scientific publications) outputs as causing a drop in its innovation ranking.

Complementing the overall GII ranking, the Global Innovation Efficiency Index shows which countries are best in transforming given innovation inputs into innovation outputs. Countries which are strong in producing innovation outputs despite a weaker innovation environment and innovation inputs are poised to rank high in this "efficiency" index. Here is the ranking:

1. China
2. India
3. Republic of Moldova
4. Malta
5. Switzerland
6. Paraguay
7. Serbia
8. Estonia
9. Netherlands
10. Sri Lanka

New Patent Offices & Courts

Today the USPTO announced plans to open regional USPTO offices in or around Dallas, Texas, Denver, Colorado, and Silicon Valley, California. These offices are in addition to the already-announced first USPTO satellite office to open on July 13 in Detroit, Michigan. The four offices will function as hubs of innovation and creativity, helping protect and foster American innovation in the global marketplace, helping businesses cut through red tape, and creating new economic opportunities in each of the local communities.

The offices announced today will help the USPTO attract talented IP experts throughout the country who will work closely with entrepreneurs to process patent applications, reduce the backlog of unexamined patents, and speed up the overall process, allowing businesses to move their innovation to market more quickly, and giving them more room to create new jobs.

"By expanding our operation outside of the Washington metropolitan area for the first time in our agency's 200-plus year history, we are taking unprecedented steps to recruit a diverse range of talented technical experts, creating new opportunities across the American workforce," said David Kappos, Director of the USPTO. "These efforts, in conjunction with our ongoing implementation of the America Invents Act, are improving the effectiveness of our IP system, and breathing new life into the innovation ecosystem."

Silicon Valley provides the USPTO with a pacific time zone hub in the heart of California's most vibrant innovation center. Silicon Valley, and the areas that surround it, contain many of the USPTO's top filers as well as legions of start-up and small tech companies that depend on the USPTO. Further, Silicon Valley's great quality of life and abundant population of engineering talent will provide fertile recruiting grounds for the Agency. The USPTO recognizes the challenges of retention in a hyper-competitive market, and will work to construct a concept of operations for the three offices that recognizes such challenges.

Meanwhile, on 29 June the European Union has finally settled on the sites for its first pan-European Unified Patent Court. The Court's Central Division of the Court of First Instance will be in Paris. Munich will be dealing with patents related to mechanical engineering, while London will handle patents related to the pharmaceutical industry and life sciences. The deal clears one of the final political hurdles on the way to a one-stop shop for patents to be granted in a single place and be valid across 25 countries.

The long-awaited decision paves the way for establishing less expensive, simpler and more efficient patent protection for businesses, especially for small and medium-sized enterprises, in the EU. Instead of applying for patent in 27 member states, businesses can now apply in one place.

The Unified Patent Court will have exclusive competence in respect of actions relating to the validity or infringement of a European unitary patent. This will eliminate the risk of multiple patent lawsuits in different member states concerning the same patent, as well as the risk that court rulings on the same dispute might differ from one member state to another. In addition, the single system will bring down patent litigation costs for businesses significantly. The European Commission has calculated that, with the single court, litigation expenses companies can be reduced by approximately 289 million euro each year.

The Unified Patent Court is part of the future unitary patent system in the EU. The other two elements are: a regulation on the unitary patent itself and a regulation on translation arrangements for that patent. The member states and the European Parliament agreed on the two regulations in December 2011.

Tomato's color uniformity: pretty or sweet

The next time you bite into a supermarket tomato and are less than impressed with the tart cardboard taste, blame aesthetics. A new study reveals that decades of breeding the fruits for uniform color have robbed them of a gene that boosts their sugar content.

Farmers pluck the fruits from the vine before they are ripe, and for about 70 years breeders have selected tomatoes that are uniformly light green at that time. This makes it easier to spot the tomatoes that are ready to be harvested and ensures that, by the time they hit supermarket shelves, the fruits glow with an even red color. Wild varieties, in contrast, have dark green shoulders, and that makes it harder to determine the right time to harvest.

In wild tomatoes, SlGLK2 (the Golden 2-like transcriptor factor behind the color change) increases the formation of chloroplasts, the compartments in plant cells that carry out photosynthesis. Chloroplasts use a green pigment, chlorophyll, to capture the sunlight plants need to grow. A higher number of chloroplasts gives wild tomatoes their darker green color.

In most tomatoes on supermarket shelves, however, SlGLK2 is inactive. While the mutation was beneficial to farmers, it's not such a sweet deal for consumers. Chloroplasts use the light energy they capture to convert carbon dioxide and water into sugars. Tomatoes with a mutated SlGLK2 gene not only have fewer chloroplasts, they also sport less sugar.

So far, this is just a hypothesis. The real culprit affecting tomato flavor could also be a production system that picks tomatoes before they are ripe.

Read the paper in Science 29 June 2012: Vol. 336 no. 6089 pp. 1711-1715 DOI: 10.1126/science.1222218.

Happy B-Day Jean-Jacques Rousseau

Today is the 300^th anniversary of Jean-Jacques Rousseau, philosopher, pedagogue, author, composer and botanist, born in Geneva on 28 June 1712. Rousseau became famous in 1750. The Academy of Dijon held a competition for academics to answer the question whether the boom of the sciences helped to "improve morality."

The answer Rousseau provided in his essay Discours sur les Sciences et les Arts shocked Europe and instantly made him famous. He won the competition with the disturbing notion that the development of civilization was in truth a story of decline and decay: in his "natural state" man lives independently and freely, but in society he is like a slave in increasingly tight chains — the evil lies in the essence of society. This provoked a scandal in this age of Enlightenment that celebrated the continuous, indeed inevitable, improvement of life by science and technology.

At age 50 he published his novel Emile on pedagogy (and the profession of a religion without a church) and his philosophical treatise Contrat Social on the reconciliation of human nature with political rule. He wrote: "The problem is to find a form of association in which each, while uniting himself with all, may still obey himself alone, and remain as free as before." Only the law should be above the individual.

The books were burnt under a parliamentary decree in the courtyard of the Palace of Justice in Paris, and on 9 June 1762, Rousseau fled from Paris to his native Geneva, where this time his books were burnt in front of the city hall and he was declared persona non grata.

Rousseau did not propose solutions, he unveiled paradoxes. His arch-enemy Voltaire summed it up with a marginal note he wrote in one of Rousseau's books: "You always exaggerate everything." It took until his 200^th anniversary in 1912 to be accepted with Dunant and Calvin as one of the embodiments of the esprit de Genève. Today he is considered a forefather of environmentalism and the occupy movements, but his main message is that we continuously fail to meet our own expectations. Society has changed: Rousseau's estate is now part of UNESCO's world heritage.

berberine CI 75160

Berberine is strongly yellow colored, which is why in earlier times Berberis species were used to dye wool, leather and wood. Wool is still today dyed with berberine in northern India. Under ultraviolet light, berberine shows a strong yellow fluorescence. Because of this it is used in histology for staining heparin in mast cells. As a natural dye, berberine has a Colour Index (CI) of 75160.

Berberine is a quaternary ammonium salt from the protoberberine group of isoquinoline alkaloids. It is found in such plants as Berberis (e.g. Berberis aquifolium (Oregon grape), Berberis vulgaris (Barberry), and Berberis aristata (Tree Turmeric)) and Coptis chinensis (Chinese Goldthread, Huang-Lian, Huang-Lien), and to a smaller extent in Eschscholzia californica (Californian Poppy). Berberine is usually found in the roots, rhizomes, stems, and bark.

Recent progress in metagenomcs (databases, mathematical algorithms, modeling approaches, and software packages for the study of the gut microbiome) has lead to the suspicion that one cause for obesity might be the lack of sufficient Coptis chinensis in the diet. Are you fat? Check your yellowness.

America Invents Act

I wrote several times, that inventions rarely come from a stroke of genius. We are pack animals, and science is a common understanding of the physical world around us, not the physical world itself. Researchers work hard to further our understanding of the world and to uncover new synergies, creating sort of an ether of knowledge. This also includes tools and technologies to further this knowledge.

From time to time, in this ether local areas condensate and great insights are born. When considered in retrospect, these insights look like strokes of genius, but they would not have been possible without a maturity attained in a certain context, or the availability of new tools.

Because of this, I like to say that ideas and inventions reach a point in time when they are ripe, and the first to recognize them is the inventor. This perspective instills a sense of urgency, because if you dither with your invention, somebody else will beat you.

In the old era of big science and large industrial laboratories, only an elite had sufficient exposure to this ether to make groundbreaking inventions. Through conferences and journals it was more or less known who works on what, and it was possible to time the filing of patents, for example so that an application is laid open the day the Tokyo Data Show opens, where a new product is unveiled.

Today, when due to the disappearance of advanced research there is less differentiation from vendor to vendor, product introductions are rarely timed to industrial fairs, but released as soon as they are ready. In computer technology for example, there are no longer a few thousand R&D personnel, but hundreds of thousands if not millions; Apple alone claims to have created 210,000 iOS app economy jobs in the US.

This huge number of technologists, combined with the increased litigation for patent infringements and the proliferation of non-practicing entities (NPE) and other patent trolls, impinges an even greater sense of urgency on protecting one's intellectual assets as fast as possible. National patent agencies support this urgency through the introduction of new processes for accelerated protection mechanisms.

Patents are not necessarily the only form of protection for one's research investment. In Switzerland, the Swiss National Science Foundation encourages patent protection mostly for inventions that are either very fundamental, or are easy to replicate. For all other inventions the recommendation is to save the money that would be spent in patent application and maintenance fees and spend it instead on the salaries for additional researchers. The reasoning is that an agile well-managed enterprise can generate new products faster than the competition can copy them, hence a defensive publication is more efficient than a patent.

In the US, the law protected the first to invent, not the first to file, so in general companies were sometimes a little slower than Swiss companies. However, this has recently changed. This picture taken September 16, 2011 at Thomas Jefferson High School for Science and Technology in Alexandria, Virginia, shows President Obama signing the America Invents Act, historic patent reform legislation that is helping American entrepreneurs and businesses bring their inventions to market sooner.

The America Invents Act helps businesses, inventors, and entrepreneurs in five immediate ways:

A fast track option for Patent Processing within 12 Months: Instead of an average wait time of almost three years, the Patent and Trademark Office will be able to offer startups and growing companies an opportunity to have important patents reviewed in one-third the time – with a new fast track option that has a guaranteed 12-month turnaround. Patent ownership is a critical factor venture capital companies consider when investing in entrepreneurs hoping to grow their business.

Reducing the current patent backlog: Under the Obama Administration, the patent backlog has already been reduced from over 750,000 patent applications to 680,000, despite a 4% increase in filings. The additional resources provided in the law will allow the Patent and Trademark Office to continue to combat the backlog of nearly 700,000 patent applications and will significantly reduce wait times.

Reducing litigation: The Patent and Trademark Office will offer entrepreneurs new ways to avoid litigation regarding patent validity, at costs significantly less expensive than going to court.

Increasing patent quality: The Patent and Trademark Office has re-engineered its quality management processes to increase the quality of the examinations and has issued guidelines that clarify and tighten its standards for the issuance of patents. The legislation gives the USPTO additional tools and resources to further improve patent quality, and allows patent challenges to be resolved in-house through expedited post-grant processes.

Increasing the ability of American Inventors to protect their IP abroad: The new law harmonizes the American patent process with the rest of the world to make it more efficient and predictable, and make it easier for entrepreneurs to simultaneously market products in the U.S. and for exporting abroad. The Patent and Trademark Office has also expanded work-sharing with other patent offices around the world to increase efficiency and speed patent processing for applicants seeking protection in multiple jurisdictions.

Mobile color selection feedback

Commercial color print workflows sometimes require manual intervention to adjust colors, for example to change a background color or the color of rules. Instead of aborting the job, sometimes the color can be changed on the fly. Typically this is accomplished with a color selection tool running on a mobile device, such as a pad computer or a smart phone.

Mobile color selection tools can use the built-in motion sensors for input. For output, due to the limited available screen space, the systems just show a swatch and maybe a color name. We describe a new output design that is well-tailored to motion-sensor based input. Using the right user interface paradigm allows users to work more efficiently, thus cutting costs and increasing profits.

Sometimes a print job requires changing a solid color, for example when the halftoning algorithm creates an unexpected interference pattern (moiré) or when the color does not print well on the particular media. It is then necessary to abort the job and send it back to the client. Such an issue can delay a job for days, possibly causing problems with the service level agreement (SLA).

The previous generation devices like laptops and tablet computers used a graphical user interface (GUI) metaphor known as WIMP, for windows, icons, menus (or mice) and pointing devices. Current mobile devices like slates and smart phones, use a different GUI paradigm known as MPG, for multi-touch, physics and gestures.

As we move beyond WIMPs, the visual feedback metaphors are no longer adequate, because they are optimized for a mouse moving on a two-dimensional plane. Current mobile devices have built-in accelerometers and gyroscopes. With them, movement between points on a plane is replaced with roll, yaw, pitch, and translational movements in three-dimensional space (see this earlier post for an informative video). In this particular implementation, the GUI consists of a colored patch and the color term.

We can provide visual feedback for MPG color selection tools using a rivet metaphor. A rivet is a short metal pin or bolt for holding together two plates of metal, its headless end being beaten out or pressed down when in place. Here are two examples of rivets. Left: Round head. Right: flat head.

Regardless of how the MPG GUI maps the Tait–Bryan angles yaw, pitch and roll, as well as translational movements and acceleration into color specifications, either absolute or relative, we use the image of a rivet to provide feedback.

The rivet head provides feedback on the total available color gamut and the hue at the center of the edge towards the user indicates the currently selected hue correlate. The length of the cylindrical shaft indicates the currently selected lightness correlate. The shaft diameter indicates the chroma correlate.

We anticipate that the user can easily learn the correspondence between MPG input and the effect on the color selection process, thus providing a very effective tool.

The figure below shows the current state of the art for WIMP operating systems. The left window is from the Windows operating system, where the color selection tool has three panels. On the bottom right is a panel showing a patch with the currently selected color and the previous selection. On the bottom left are the RGB counts. The top left is the graphical color selection panel.

Regarding color selection tools in applications, they mostly use the tool provided by the operating system. One notable exception is the color selection tool in Photoshop. This application actually has two color selection tools, one simple and fast for the experienced user, and one more detailed for careful color selection. We describe first the simple tool at the left of the figure below and then the detailed tool at the right.

The simple tool has three panels: a sample patch (actually two: one each for foreground and background), a set of three sliders for RGB counts, and a complex graphical single point selection panel. The single point panel is a rectangle where the abscissa is a correlate for perceived hue. The ordinate is a correlate of saturation, which contains both lightness and chroma. This hue–saturation paradigm allows color specification through a single two-dimensional point, at a possibly increased cognitive cost.

The larger window at the right side of the above figure is Photoshop's detailed color selection tool. On the top right we note the split old and new color patches as in the Windows tool. The slider in the middle is used to select the hue, while the chroma and lightness are selected in the large square at the left. In this square, the abscissa is a correlate of chroma, while the ordinate is a correlate of lightness.

All these tools have evolved from the early days of color GUIs and are optimized for input on a two-dimensional surface with a mouse. The slider interfaces are actually older, when the early interactive graphics workstations had dials for data entry.

Like everywhere else in our lives, also on the print shop floor we transitioned from desktop and laptop computers to mobile devices like pads and smart phones. Parallel to this transition is the paradigm shift from WIMP to MPG GUIs and the visual feedback for three-dimensional input devices must be different from that for two-dimensional devices.

We use a feedback mechanism in the approximate shape of a rivet, which can change its location and orientation in space in accordance to user gestures. This mechanism changes its appearance according to the color being selected.

The rivet head represents the color gamut. It can be either a flat head showing the gamut in a chromaticity diagram, or a three-dimensional head showing the full gamut (with transparency to optionally mark the currently selected color). When users change the hue through a gesture, the rivet rotates along its symmetry axis, so that the currently selected hue is always pointed towards the user.

The shaft is used to represent chroma and lightness. The shaft length is proportional to the current selection's lightness and the shaft diameter is proportional to the current selection's chroma. The shaft itself is colored in the current selection's color.

Yaw, pitch, and roll, also known as Tait-Bryan angles, named after Peter Guthrie Tait and George H. Bryan, are a specific kind of Euler angles used to define the relative orientation of an object with respect to some reference orientation, usually a set of reference axes. The three angles specified in this formulation are defined as the roll angle, pitch angle, and yaw angle. Yaw, pitch and roll are used in mobile devices where the object in question is the handheld device itself.

The figure below illustrates the Tait-Bryan angles. They can be statically defined using a line of nodes constructed by the intersection of two non-homologous planes (for example XZ and xy are not homologous planes), unlike proper Euler angles which use homologous planes (for example XZ and xz).

This second kind of Euler angles is such as it is equivalent to three rotations composed with a different axis, z-y-x for example. There are therefore six possibilities of this kind (xyz, xzy, zxy, zyx, yzx, yxz). They behave slightly differently than Euler angles. In the zyx case, the two first rotations determine the line of nodes and the axis x, and the third rotation is around the axis x.

Because the line of nodes is the intersection of two non-homologous planes the pitch angle is measured from the horizontal plane instead of the vertical axis. Therefore this kind of construction would give a pitch equal to zero for an airplane flying horizontally while the first kind of Euler angles would assign it an angle of π/2.

Since this terminology originates in aeronautics, in this section we use an aircraft instead of a handheld mobile device, but the physics is the same. The concepts are shown in this figure:

Yaw, pitch and roll are used in aerospace to define rotations between a reference axis system (world frame) and a vehicle-fixed axis system (body frame), which in the context of an aircraft sometimes are called its heading, elevation and bank.

Consider an aircraft-body coordinate system (body frame) with axes XYZ which is fixed to the vehicle, rotating and translating with it. This intrinsic frame of the vehicle, XYZ system, is oriented such that the X-axis points forward along some convenient reference line along the body, the Y-axis points to the right of the vehicle along the wing, and the Z-axis points downward to form an orthogonal right-handed system.

Consider a second coordinate system (world frame) with axes xyz, aligned having x pointing in the direction of true north, y pointing to true east, and the z-axis pointing down, normal to the local horizontal direction.

Given this setting, the rotation sequence from xyz to XYZ is specified by and defines the angles yaw, pitch and roll as follows:

• right-handed rotation Ψ ∈ (-180, 180] about the z-axis by the yaw angle
• right-handed rotation θ ∈ [-90, 90] about the new (once-rotated) y-axis by the pitch angle
• right-handed rotation φ ∈ (-180, 180] about the new (twice-rotated) x-axis by the roll angle

The motion of an aircraft is often described in terms of rotation about these axes, so rotation about the X-axis is called rolling, rotation about the Y-axis is called pitching, and rotation about the Z-axis is called yawing.

The equivalent MPG feedback to the WIMP feedback in the above figures is shown on the top side of the figure below. In this example we use the flat head rivet metaphor from the right side of rivet figure above. It is obvious how to generalize to the three-dimensional gamut as mentioned earlier.

the user has selected a dull green

the user has rotated the hue towards yellow and increased both lightness and chroma

In this case the gamut is a chromaticity diagram, referring to a ZR-class HP display. Since CIELAB does not have a chromaticity diagram, we use the CIELUV space. For a most sophisticated feedback, a color appearance space like CIECAM02 could be used, rendered with transparency to show the position of the mark for the selected color. For a simpler feedback, the rivet could be displayed in a pure frontal projection, with a hue ribbon lining the flat head edge. In the higher figure above, the rivet is rotated along the symmetry axis so that the same green as in the color selection tool figures at the beginning is in the front.

The shaft is colored in this green, its diameter corresponds to the chroma and the length to the lightness.

The lower of the above figure shows what happens when the user moves the mobile device to select a yellow color. The head rotates so that yellow is now on front. The shaft is colored in this yellow and has become longer and thicker. The left and right sides of the figure are at the same scale.

In summary, I have described how the feedback for color selection tools for a WIMP GUI are not adequate for mobile devices, which make extensive use of MPG GUIs. I have presented a rivet metaphor that provides a much more ergonomic representation of the color selection tool's state. This radically new representation allows print shops to work faster and more reliably when they have to select or modify colors, thus preventing a disruption of the workflow.

Providing a commercial print workflow with superior tools will cut costs due to workflow exceptions and maximize the press owner's profits. A successful customer will buy additional presses from the vendor and use more consumables.

Path rendering on the GPU

Archimedes of Syracuse famously claimed "give me a place to stand on, and I will move the Earth." Understanding the principle of the lever is key for being successful in technology.

Isatis tinctoria

Isatis tinctoria is a flowering plant in the family Brassicaceae, from whose leaves a blue dye is produced (the flowers are yellow). It is commonly known as woad or Asp of Jerusalem. The common names in Italian, German, and French are guado or gualdo, Färberwaid, respectively pastel des teinturiers or guède. It was used with weld (for yellow) and madder (for red) to produce full color images from at least the Neolithic Age, when it was introduced in Europe.

With the European discovery of the seaway to India, woad was replaced with indigo or Indigofera tinctoria, which contains the dye in higher concentration. Today, synthetic indigoes are used. However, Isatis tinctoria still has use in some inkjet inks because it is biodegradable.

Color and Motion

From humans to insects, color and motion information are thought to be channeled through separate neural pathways for efficient visual processing, but it remains unclear if and how these pathways interact in improving perception of moving colored stimuli. By using sophisticated Drosophila genetics, intracellular electrophysiology, two-photon imaging, and behavioral experiments, Trevor Wardill et al. found that early in the processing stage, color photoreceptors influence the motion pathway and that this input improves the flies' optomotor performance in a flight simulator.

Read the paper Multiple Spectral Inputs Improve Motion Discrimination in the Drosophila Visual System in Science, 18 May 2012: Vol. 336 no. 6083 pp. 925-931 DOI: 10.1126/science.1215317.

do not fear

Words displayed in large fonts elicit stronger emotional responses, according to a new study. Mareike Bayer, Werner Sommer, and Annekathrin Schacht measured brain activity in 25 adults while showing them 72 emotionally positive, negative, and neutral words. They included words for gift, death, and chair. The team displayed the words in either 28-point or 125-point Arial font. Volunteers displayed stronger emotion-related brain activity 10 milliseconds earlier for the larger font size versus the smaller one, the authors reported online yesterday in PLoS ONE (open access). What's more, emotional signals elicited by the larger font size lasted a total of 180 milliseconds longer. The results are similar to emotional responses to large and small versions of pictures with fearful, disgusting, or sex-related content. Pictures hold biological relevance for people, since a big photo of a predator probably signals proximity to you. Similar emotional effects on font sizes probably reflect the importance language holds in our society, the authors speculate.

Lou Silverstein

In the mid-80s, when we were building up the new digital color research area at PARC, we invited as many color scientists active in digital reproduction as we could. We had two motives: scope the field and identify potential candidates to hire.

One visitor stood out for his deep knowledge both in color vision science and in the mathematical modeling of reproduced color. With his tall slender figure he was immediately recognizable, and his soft voice was very determined in technical discussions. He also had a unique curriculum vitæ: he developed the avionics color displays for Honeywell and Boeing while never changing location; his employer would change, but his lab remained intact. While others were trying to characterize CRTs, he build a complete model for LCD displays; without him, we would not have high fidelity color LCD displays today.

Dr. Louis D. Silverstein has died at age 61. Farewell Lou!

color semiotics survey

I has been almost a year since we posted on Maryam's color semiotics research.
Everyone is talking about it these days, but where can we find a certain rule or framework which defines it? What parameters are involved? Is this framework useful for designers? Can it be communicated? Can its variation be modelled?

Last January 23^rd Maryam reached the 1800^th response to her survey. This is the last chance for your participation before her ultimate analysis. Please take to survey now at https://www.keysurvey.co.uk/votingmodule/s180/survey/365495/1a02/

processing retinae

We know very little about the physiology of color vision. For example, in the case of dichromatic color vision, in the past the trick was to have a rolodex of unattached dichromates, volunteering as an emergency physician, and when an expired dichromat showed up in the ER, there was an hour's time for wet color science.

Recently, a lot of progress has been made in color vision physiology by leveraging state-of-the-art equipment. A couple of months ago we reported on Kathy Mullen's breakthrough leveraging a new fMRI scanner in Australia. Today we cross the big pond and look at an application of big data technology.

Michele Fiscella harvests the retina of a mouse, places it on a MEA chip, sprinkles it with a nourishing fluid, and for the couple of hours the retina remains functional, projects patterns on the retina and captures the interneural traffic.

MEA—for Micro Electrode Array—is a technology from the lab of Prof. Andreas Hierlemann. On a surface of 3.6 mm², 11,011 elliptical microelectrodes probe the retina. On the average, for each neuron there are 14 microelectrodes, and each one delivers 20,000 measurements per second. The hope is to sleuth what information is passed bottom up from the retina to the LGN.

To do that, the retina has to be placed upside down on the MEA, i.e., with the retinal ganglions on the microelectrodes and the photoreceptors towards the stimulus source. This is not necessarily contra naturam, as there is nothing coming top down from the LGN, which is not part of the experiment.

Analytical engines, here the big data come…

color consolidation

In a recent press release, Danaher Corporation on 2200 Pennsylvania Avenue, NW in Washington, D.C. announced that it purchased X-Rite for approximately $625 million. X-Rite had purchased Pantone and Gretag, and the latter had purchased Macbeth. Danaher already owns well-known brands like Tektronix, Fluke, Leica Microsystems, Beckman Coulter, Videojet, and Esco (a leader in industrial digital printing) among others.

imagination and vision

In last month's post on conscious awareness I mentioned Kathy Mullen's work using an fMRI machine in Australia. Although that machine is one of the finest available, it is still a blunt instrument, but that it is all we can do with humans, where we are limited to non-invasive procedures.

A Digital Digit Camera

Color Blindness Simulator

Color Oracle is a free color blindness simulator for Windows, Mac OS X and Linux. It claims to take the guesswork out of designing for color blindness by showing you in real time what people with common color vision impairments will see.

Related posts:

• The First Time Ever I Heard Your Face
• The Colorful Blind Painter

Conscious awareness

When in the late Eighties I first compiled an introductory color tutorial, I spent considerable time to come up with a diagram illustrating that vision is not hierarchical, i.e., the simple "camcorder model" of a distal event followed by a proximal stimulus which is in turn followed by a brain event is incorrect, because there is no homunculus in our head looking at the brain event.

In this diagram—for which, by the way, I received quite a bit of negative criticism—this is indicated by making the arrows in the turquoise lines bidirectional, and I used to say, "at every level there is feedback." The most critical people were those who believed perception is strictly bottom-up.

Besides of the homunculus paradox, there has long been physiological evidence that vision has a top-down component. For example, the diameter of the center-surround fields at the cone level in the retina is modulated by a fountain of chemicals, which is global and therefore controlled by higher up stages. In fact, there are about 15 different hormones implicated in retinal fountains. More in general, chemical messengers (peptides) are more important than electrical signals for propagating information in the body (see Susan Greenfield).

At the Color Imaging Conference in San Jose last November, Kathy Mullen presented some very interesting new results obtained through fMRI studies. For example, much more information flows from the cortex to the LGN than from the retina to the LGN.

As all the cycles on my PC are consumed by a long backup, I am catching up with my reading. I am looking at Kaspar Meyer's perspective Another Remembered Present on page 415 of Science volume 335 (27 January 2012).

First of all, I had the direction of the feedback backwards. If we have a mental model of reality and use the visual system to confirm or adjust it, then the feedback is from the bottom in direction up, not the other way as I was saying.

Meyer writes that "A particularly intriguing observation is that while the initial bottom-up activation sweep along the sensory pathways can accomplish stimulus processing of considerable complexity and yield certain automated behaviors, conscious awareness of a sensory object appears to depend on top-down signals."

After presenting two influential theories of consciousness, Meyer cites some recent physiological findings and concludes: "Taken together, such data suggest that top-down signals, contrary to their common designation as 'feedback signals', have more than a modulatory function: They can reconstruct neural representations of considerable resolution in the early sensory cortices. Why would the conscious mind be grounded in dispositional records held in convergence-divergence zones, rather than the 'raw' version of reality initially established in the early sensory cortices through bottom-up signals from the thalamus? One potential answer is tied to prediction: The brain 'constantly and internally [generates] varieties of hypotheses and [tests] them upon the outside world, instead of having the environment impose (instruct) solutions directly upon the internal structure of the brain'."

Local optimization

I received a question about the paper Assessing color reproduction tolerances in commercial print workflow mentioned in a recent post. The interlocutor asks why I bother creating custom color scales, instead of just using the Farnsworth's 100 Munsell hues: the implementation would be much simpler.

I believe this question is a nice example of the difference between a color engineer and a color scientist. Let me explain:

Shining Silver Surfer in the Snow

What do you do with several hundred LEDs? You make a movie like this:

Easy, right? After all, LEDs work better at low temps. Well, maybe not.

Why Doesn't Your Camera Look Like a Telescope?

The visual balance between globality and interactions

Lucia Ronchi's book The visual balance between globality and interactions is available via open access from Google Books from this link: http://books.google.com/books?id=HZDUqzJ7er8C.

This is the foreword:

This review on the current state of experimental visual research mainly concerns the response to complex test objects, involving the conjunction of several stimulus features, followed by the activation of various mechanisms, hierarchically located at different levels along the visual pathway. The prediction of the response is complicated by the underlying jungle of mutual interactions, as shown by the recent modelling. From the stand point of the laboratory experiment, the traditional response indexes need to be flanked by the so called «global" responses. A glance to the literature appeared during the past two decades reveals that this topic is not new, however is seems to us that it is yet at the specialized levels, and an educational simplification would be needed.

The present preliminary attempt along these lines is divided into two parts. First of all, our experimentation performed during the past eleven years is described, by using an assumed global index, the visual balance, per se not new, because it has been discussed as one of the ingredients of perceptual harmony during the past two centuries. In the second part, the plethora of concepts underlying the concept of balance and its visual process are summarized, after a glance to the more-or less recent visual literature.

The command to Download the PDF is hidden under the cogwheel icon.

Assessing color reproduction tolerances in commercial print workflow

The presentation of this paper was somewhat hasty, because I forgot to finish the slides. I only realized this while I was setting up my laptop and quickly thumbed through the slides. I only had the short time during the break to quickly assemble the presentation by copying chunks from the paper, while also trying to help Dr. Tastl who was having a problem getting PowerPoint to recognize the projector. I guess this is what happens when we are burnt out…

Harmonious colors: from alchemy to science

Last weekend three storms swept through the Bay Area and it was a good thing, because it cleaned up the very dirty air. The attendees of Electronic Imaging enjoyed a perfect weather.

The First Time Ever I Heard Your Face

Neil Harbisson is a color-blind (achromatopsia) artist from Spain who has a cybernetic "third eye" attached to his head that enables him to hear color. This also makes him a genuine cyborg. Since 2005, Harbisson has used his sonochromatic retina to design sound portraits of famous people. The first person he experimented on was the immediate heir to the British throne, HRH The Prince of Wales.

He scans the eyes, hair and skin color of his subjects and works the individual notes into a musical chord. Harbisson explained, “Prince Charles noticed my electronic eye so, I politely asked him if I could listen to his face.” Apparently Prince Charles' lips sounded like a high E [a blue blood? --njg]. But his hair was almost inaudible—musically thin due to male-pattern baldness.

Some notable quotes from a recent BBC Outlook interview:

• Color as words are "really, really, really, impossible to understand"
• "Color has a frequency that we can't hear"
• "Red is between F and F^#" [? I would've thought it was A/A^#. --njg]
• "I hear the color through my bones, not through my ears"
• The BBC interviewer's voice is "between orange and red"
• Neil's fave "color" is Aubergine, which has a high pitched tone
• "White is silent" [? I would've thought that was black. --njg]

And finally, here is Neil with his Eyeborg on daily maneuvers (video).

Related post: The Colorful Blind Painter

R&D not sufficient for success

As a follow-up on our previous post, the Economist has an interesting blog post Gone in a flash, which delves in the problem that for a technology company research is necessary but not sufficient for success, as I had pointed out in an earlier analysis of the fruits of R&D investment over a long time period.

An interesting comment in the Economist's blog post is that mergers and acquisitions are not necessarily useful. As Geoffrey A. Moore wrote in his latest book, the acquisition of a company is only fruitful if the buyer already has knowledge of that company's technology, so it can digest it.

If R&D is necessary, is it sufficient that a company must be able to digest research to feed its business? Digesting research is very difficult; maybe this is why companies fail over time: they are unable to find the leaders with this skill.

Why is Kodak near death while Fujifilm is thriving?

Fascinating perspective in The Economist. Here are some quotes:

Fujifilm, too, saw omens of digital doom as early as the 1980s. It developed a three-pronged strategy: to squeeze as much money out of the film business as possible, to prepare for the switch to digital and to develop new business lines.

Kodak had become a complacent monopolist. Fujifilm exposed this weakness by bagging the sponsorship of the 1984 Olympics in Los Angeles while Kodak dithered. The publicity helped Fujifilm’s far cheaper film invade Kodak’s home market.

Another reason why Kodak was slow to change was that its executives “suffered from a mentality of perfect products, rather than the high-tech mindset of make it, launch it, fix it,” says Rosabeth Moss Kanter of Harvard Business School, who has advised the firm.

Hindsight is always 20/20 but that last quote is reminiscent of the difference in business philosophy between Microsoft and Apple. One wonders what Steve Jobs might have said about that. We do know what Bob Lutz (former Vice Chairman of General Motors) thinks about Ivy League business schools and MBA spreadsheet-based business strategies.

Postscript: See the Comments below for additional perspective.

Crows Remember Colors For A Year

Researchers have recently found that crows possess a long-term memory that allows them to remember colors for at least a year, a talent they use to successfully select containers containing food in experiments based on color cues after extended intervals. "It is not easy even for human beings to remember visual color information for a year. Crows may be even better than human beings in a certain aspect of memory," said Shoei Sugita, a professor of animal morphology at Utsunomiya University who led the joint research with Chubu Electric Power Co. The finding came as part of a study that Chubu Electric, troubled by problems presented by crows' nests on power line towers, commissioned in 2008.

Source: Science & Technology News from Japan, December 2011 • Dr. Felix Moesner

Colored Time

"I have a meeting at purple o'clock." Say what!?

Meet the TimeHue® clock that represents time in HSL color space.

Frankly, I don't get it. Looks to me like a huge impedance mismatch for cognition. I'd have to keep checking my watch to decode it.

Finding structure in large data sets

In color science we do not have sufficient knowledge about the visual system to formulate analytical models. However, in the past two centuries we have accumulated sufficient knowledge, that we do not have to resort to machine learning approaches like hidden Markov methods either. We are able to come up with equations that correlate well with what is perceived.

One would then assume, that color scientists excel in the mathematical statistics branch of correlations. Alas, the practice is less glorious and we tend to rely on Pearson's correlation to gauge the strength of association between pairs of stochastic variables. The caveat is that Pearson's correlation captures only linear correlation.