Rival, predator, mate: Mapping the molecules that detect chemical cues. By Catherine Dulac

(Medical Xpress) -- The chemical cues that signal animals’ identity are so important to letting other individuals know how to behave in the presence of a member of their own species – whether to mate or fight, for example -- that most mammals have a sensory organ dedicated to detecting pheromones.

The vomeronasal organ (VNO), a tubular structure located in the nose of most mammals, has long been assumed to be primarily devoted to pheromone detection. But when Howard Hughes Medical Institute (HHMI) researchers mapped more than 100 different receptor molecules in the VNO to the particular scents that they perceive, they discovered that most VNO neurons and receptors are dedicated to the detection of animals from other species. By studying the response of neurons in the VNOs of mice to chemical cues collected from not just mice, but also insects, snakes, birds, and an assortment of other species, the research discovered that the VNO is sensitive not just to members of the same species, but also to predators and competitors.

“This tells us that the animal doesn't need a lot of receptors to identify another mouse, but it needs a lot of receptors to be able to detect many, many types of predators," said Catherine Dulac.

The research was led by Dulac, an HHMI investigator at Harvard University who wants to understand how animals detect, identify, and process sensory signals. Dulac and her colleagues from Harvard University published their findings in the September 22, 2011, issue of the journal Nature.

"The VNO has been highly advertised as the sensory organ that detects pheromones -- cues exchanged between members of the same species," Dulac says. But the new findings indicate that the VNO plays a much broader role. In fact, in Dulac’s team’s studies of the mouse VNO, the number of sensory receptors dedicated to pheromone detection was found to be greatly outnumbered by those that perceive cues from other species.

Packed into the VNO are tens of thousands of neurons, each one equipped with receptors for a particular chemical cue. When a pheromone or other chemical cue arrives at the VNO and binds to its matching receptor, the neuron is activated and sends a signal to the brain. The first receptors in the VNO were discovered by Catherine Dulac herself together with HHMI investigator Richard Axel more than 15 years ago, and scientists now know that they come in approximately 300 different versions. Prior to this study, however, scientists did not know the full range of chemical signals the organ could detect. “We had no idea what receptor was doing what,” Dulac says.

Dulac thought a more comprehensive survey was needed. "We wanted to know the specific molecular identity of receptors activated by a given animal signal, something that nobody had been able to achieve so far,” she explains. So, together with her postdoctoral Associate Yoh Isogai, she turned to nearby facilities that house live animals – including the Harvard Museum of Natural History, the Museum of Science, and the New England Wildlife Center – and requested bedding from the cages of various species -- birds, snakes, alligators, ferrets, and more. Bedding material absorbs a cocktail of biological chemicals, including those in urine, feces, saliva, and other gland secretions, and exposure to particular animals’ bedding can elicit behavioral responses in mice, such as defensive behaviors or changes in eating. For aquatic animals, such as alligators, fecal pellets were collected. Live insects were also included. Finally, the team added bedding from both male and female mice. In all, Dulac’s team collected 29 different chemosensory cues to expose to the mice in their experiments.

In previous studies, researchers had correlated different signals to the activation of neurons in the VNO. But they had lacked the tools to identify the specific sensory receptor in the activated neurons. Rather than only measuring electrophysiological activity to or watching for the influx of calcium that indicates a neuron has been activated, as done in previous studies, Dulac and her team looked for the activation of certain genes as an indicator of neuron activity. This allowed them to simultaneously determine which receptor was expressed in those cells using 200 molecular probes developed for this purpose. Each probe would bind to the RNA that encodes a particular receptor. If that RNA was present in a particular cell, they knew the gene for that receptor had been turned on and the cell was manufacturing the protein. They used these probes to identify which receptor genes were expressed by the neurons in the VNO that responded to the chemicals present in the bedding of particular animals.

Some of the cues they tested came from animals who presented no known challenge to mice – they were neither competitors nor predators. Bedding from these animals – a woodchuck, for example – elicited no apparent response from the sensory neurons in the VNO. Among the cues that did trigger a response, the researchers were surprised to find that the majority of the VNO neurons were activated by predator cues, and not by mouse cues. They uncovered 28 receptors that responded to mouse cues, all found in neurons clustered together in a small segment of the VNO. Most responded selectively to only male or female cues. In contrast, 60 different receptors were activated in response to the non-mouse cues the team tested.

"This tells us that the animal doesn't need a lot of receptors to identify another mouse, but it needs a lot of receptors to be able to detect many, many types of predators." Some of the predator-activated receptors were very specialized, activated only by a particular snake, or bird of prey, or mammalian predator. Others were more generalized, responding to an entire class of predators, such as all snakes. Some were even more generalized and detected all predators.

Another surprising finding was the receptors are extremely specific. "We rarely saw receptors that were activated by both predator and mouse cues," said Dulac. This suggests that the activation of a single receptor is sufficient to determine the identity of the animal giving off a chemosensory signal. Dulac likens it to a switchboard, where the switch you push tells you very precisely what type of animal is being encountered.

Dulac says the importance of this work is twofold. In addition to knowing exactly what signals the VNO detects and the identity of the receptors corresponding to these signals, scientists now have the tools to manipulate this system to learn more about how the brain processes sensory information. She is particularly curious about how these receptors enable learning and identification. For example, she would like to know whether prior encounters with a signal are necessary to recognize it, or if animals innately recognize the chemical cues associated with predators and members of one's own species?