The signals from the cones are sent through a network of other nerves.
But the cones are just the beginning of the story. The eye, like any optical device, is not perfect. A point in outer space is not imaged onto the retina as a true point, rather into a somewhat more diffuse area called a blur circle. Other chromatic abberations cause colored fringes, and as people age, their lenses yellow considerably. Additionally, the light source can vary color dramatically: a sun set and a flourescent light have very different hues.
Our eye has evolved to compensate for these many flaws. Our nerve cells compare the signals from the cones, to measure the amount of green-or-red and blue-or-yellow, particularly along edges. In the diagram below, the signal begins with the cones, on the right.
The rods and cones line the back of the retina. In front of them are three types of nerve cells: bipolar cells, horizontal cells, and amacrine cells. Bipolar cells receive input from the cones, and many feed into the retinal ganglion cells. Horizontal cells link cones and bipolar cells, and amacrine cells, in turn, link bipolar cells and rentinal ganglion cells. There are about 1 million ganglion cells.
Why all the fuss? The purpose of the ganglion cells is not fully known, but they are involved in color vision. Ganglion cells compare signals from numerous of cones.
The ganglion cells add and subtract signals from many cones. For example, by comparing the response of the middle-wavelength and long-wavelength cones, a ganglion cell determines the amount of green-or-red. Moreover, they are excited by middle of their field, and inhibited by the surround, which makes them particularly sensitive to edges.
The result of all these steps, for color vision, is a signal which is sent to the brain. There are three signals, corresponding to the color attributes. These are: