Emergency
Standard
- Procedure for a planned warmup
- Procedure for a planned cooldown
Emergency Cryogenic Procedures
NIRSPEC is cooled continuously by use of Closed-Cycle Refrigeration (CCR). A compressor unit in the K2 machine room provides pressurized He gas to two cold heads on-board NIRSPEC. If this system is inoperable for more than about 15 minutes, then NIRSPEC may require a lengthy 10 day cooling cycle to become operable again.
1. Loss of CCR Cooling: Emergency Procedures
2. Loss of CCR Cooling: Vacuum Procedure
3. Reading Cryogenic Temperatures
4. Example of the e-mail alarm message
Standard Cryogenic Procedures

Planned cryo cycles on NIRSPEC require some coordination between summit and HQ staff, to ensure that everything gets done in the right order. The steps below include suggestions as to who can carry them out most easily, but these are only suggestions. Experience shows that the whole process is usually smoother when both summit and HQ staff are involved.
1. Procedure for a planned warmup
  1. Move NIRSPEC into stow if not already there.
  2. Lock out unwanted activity. (For Instrument Scientist) Log on to the HQ workstation hanauma as user nirspec and change the password using the unix command yppasswd. This will prevent random astronomers from attempting to start the NIRSPEC software or move motors while the instrument is not at thermal equilibrium. Be sure to make a note of the new password, so you can reverse the change later! Send an email to the "instrument" alias to announce that you have done this.
  3. Configure the software on waimea. (IS) Log in to the instrument host waimea as user nirspec and perform the following steps:
    1. Make sure waimea is healthy. Check the time since the last reboot with the unix command uptime. If it's been more than a week, reboot waimea before proceeding. This will help prevent crashes that would interrupt temperature logging while NIRSPEC is warming up.
    2. Get the server running. Check the status of the server with the recover script, and follow its instructions.
    3. Start temperature logging. In the home directory of the nirspec account on waimea issue the command:
      - logcryo > logs/cryotemps.warmup.[date] &
      to start the temperature logging process. This will record the temperatures on all the dewar sensors, every five minutes, in the file you just created. For the [date] part of the filename, use some identifier that will include the day, month and year such as 23jul01. Don't sweat it if you forget this as temps are logged by a cron all the time.
    4. Turn up the Aladdin detector temperature setpoint. To protect the Aladdin detector from condensation, it must not ever be the coldest point in the system. The normal temperature setpoint is 30 Kelvin. Set it up to 300K with the following command:
      - m change.detector.temp=300
      The detector block heaters do not have enough power to take it to 300K, but setting the temperature to 300 will run the heaters flat out, which will keep the Aladdin about 10-15 degrees K warmer than the rest of the system.
    5. Silence the cron jobs. When NIRSPEC is cold, there are cron jobs running which will send warning emails out if they see the temperature going up. Edit the crontab and comment out the line which has nirspec_watch in it. The editor is vi.
      - crontab -e
    6. Send email to nirspec_info. Send out an email describing what is going on, and that you have disabled the cron jobs which monitor the NIRSPEC dewar temperatures.
  4. Touch base. (IS) Contact the summit Instrument Technician on duty to confirm that all the steps above have been completed, and that NIRSPEC is ready for warmup.
  5. Check the vacuum. (For Instrument Technician) Turn on the Varian SenTorr vacuum gauge controller, and after it stabilizes make a note of the vacuum reading on the "high vacuum" channel.
  6. Turn off red, then black switches on CCR compressor. Tag out the compressor (IT) In the Keck 2 mechanical room, switch off the CCR compressor No. 3 that serves NIRSPEC. Immediately advise the responsible Instrument Scientist as to the time the CCRs were turned off, and as to the vacuum reading on the Edwards gauge beforehand.
  7. Disconnect the CCR lines at the instrument
  8. Disconnect the CCR head power at the instrument
  9. Place pump station at instrument in case of need
  10. Monitor and Record pressures occassionally during warm up Use this record to guide decision whether to pump or not.
  11. NOTE: this step should be considered only in extreme cases where the dewar needs to be warmed up very fast, since it involves some risk of damaging the detector and optical components within the dewar. if you have not been trained on this next step, consult with instrument engineer for training before proceeding! Backfill with dry nitrogen gas. (IT) First confirm that all temperatures inside the dewar are above 100 Kelvin by either calling the responsible Instrument Scientist, or checking the latest temperature log file. If all temperatures are high enough, carefully bleed dry lab-grade nitrogen gas into the dewar for just a few minutes. Do NOT push in gas until the dewar is at ambient atmospheric pressure! Stop while there is still some audible hissing coming from the dewar vacuum valve. If the dewar is at ambient pressure at 100 K, it will be substantially overpressured by the time it reaches 273 K (typical dome ambient temp), and that is not good.
2. Procedure for a planned cooldown
  1. Pump for as long as practical. (IT) Before cooling down, pump the dewar for 4-5 days if possible.
  2. Bake the getters. (IT) Make a note of the vacuum reading on the pump (and dewar if possible) before beginning this step. During the first day of pumping, connect a DC power supply to the banana jacks on the rear module leg (by the vacuum valve) to power the heaters in the getter material on the cold heads. The heaters are two 25-ohm resistors wired in parallel adjust the power supply so that the total DC power supplied to both heaters (voltage times current) is about 12.5 watts or 6.25 watts per resistor. Make a note of the vacuum reading on the pump (and dewar if possible) after the heaters have been on for at least a half-hour.
  3. Turn off getter heaters. (IT) Run the getter heaters as described above for only about 24 hours, then continue pumping.
  4. Decide when to start CCRs. (IS) This decision will usually be dictated by the observing schedule. It's best to allow 12 full days of CCR operation before the first science night. Subject to that constraint, pump the dewar for as long as practical before starting the CCRs.
  5. Lock out unwanted activity. (IS) If this has not already been done as part of the warmup, log on to the HQ workstation hanauma as user nirspec and change the password using the unix command yppasswd. This will prevent random astronomers from attempting to start the NIRSPEC software or move motors while the instrument is not at thermal equilibrium. Be sure to make a note of the new password, so you can reverse the change later! Send an email to the "instrument" alias to announce that you have done this.
  6. Configure the software on waimea. (IS) Log in to the instrument host waimea as user nirspec and perform the following steps:
    1. Make sure waimea is healthy. Check the time since the last reboot with the unix command uptime. If it's been more than a week, reboot waimea before proceeding. This will help prevent crashes that would interrupt temperature logging while NIRSPEC is warming up.
    2. Get the server running. Check the status of the server with the unix command ct, and verify that you see three server processes running. If you see none, start the server with the command runserver. If there are 1 or 2 processes running, or if the server won't start normally, see the troubleshooting page for help.
    3. Start temperature logging. If you had to restart the server in the previous step, then any cryo logging that was running before is now dead. Check for any lingering processes with the command:
      - psgrep logcryo
      Kill any suspended logcryo jobs found. If there is not a working logcryo job running, start it. In the home directory of the nirspec account on waimea issue the command:
      - logcryo > logs/cryotemps.warmup.[date] &
      to start the temperature logging process. This will record the temperatures on all the dewar sensors, every five minutes, in the file you just created. For the [date] part of the filename, use some identifier that will include the day, month and year such as 23jul01.
    4. Turn up the Aladdin detector temperature setpoint. To protect the Aladdin detector from condensation, it must not ever be the coldest point in the system. The normal temperature setpoint is 30 Kelvin. Set it up to 300K with the following command:
      - m change.detector.temp=300
      The detector block heaters do not have enough power to take it to 300K, but setting the temperature to 300 will run the heaters flat out, which will keep the Aladdin about 10-15 degrees K warmer than the rest of the system.
  7. Touch base. (IS) Contact the summit Instrument Technician on duty to confirm that all the steps above have been completed, and that NIRSPEC is ready for cooling down.
  8. Check the vacuum. (IT) Turn on the Varian SenTorr vacuum gauge controller, and after it stabilizes make a note of the vacuum reading on the "high vacuum" channel. Also note the vacuum reading on the pump. Touch base with the responsible Instrument Scientist to confirm that the vacuum is good enough to restart CCR cooling.
  9. Stop pumping. (IT) Close the dewar vacuum valve, and turn off the pump.
  10. Start the CCRs. (IT) In the Keck 2 mechanical room, switch on the CCR compressor No. 3 that serves NIRSPEC. Immediately advise the responsible Instrument Scientist as to the time the CCRs were turned on, and as to the vacuum reading on the Varian SenTorr gauge controller before and after.

Emergency Cryogenic Procedures

Standard Cryogenic Procedures