[Sciserver-users] MPE SciServer - July 2021 update

[Jonas Haase]

Dear MPE SciServer user

This is a quick email to let you know that the images for the compute containers have been updated. 
In the list in the bottom of this email you can see the new and updated software in the various images - they will be available to you when you start a new container. 
Please let me know if you experience any problems.

Some of you may have experienced issues using the compute containers in the previous week. The reason for that was some intensive use by a number of users in parallel which put a bit of a strain on the compute nodes and made them unreachable. 
We are now about 100 users on the SciServer and it is being used more and more for long-running data processing jobs. 
I would like to ask you to please be mindful of the available compute resources and not to start CPU intensive tasks with more parallel threads than there are cores available in a container (8-12 depending on where you are).

Do also try to see if you can run larger tasks inside Compute Jobs - the queueing system should even out the load when things get very busy.  
There is a example notebook and python script in 
https://sciserver.mpe.mpg.de/dashboard/files/datavolumes/4/ComputeJobs
which shows you how you conveniently can queue jobs using the SciServer API. 

We are currently evaluating the options for extending the compute capabilities on the SciServer and would appreciate any feedback you can give on your current experience and perhaps also your need of compute resources in the intermediate future (in particular if you can put them in terms of frequency of use, CPU cores and memory needed).



Updates to compute images:

Common (SciServer image)
- added Perl 5, cpanm, Astro::SIMBAD
- Installed Montage 6.0
- Update to Conda 2020.11
- Added Python modules beautifulsoup4, emcee, HCIplot 

eROSITA (xray and esass images)
- Both images should now resolve the environment for HEASoft & eSASS correctly in Compute Job containers - batch jobs can now be run there without extra steps. 
- HEASoft updated to 6.28, heasoft.py 0.1.21 
- CIAO to version 4.13
- SIXTE version 2.6.3
- the CALDB holdings in the xray_aux Data Volume are (hopefully) up to date again.
- The CALDB environment variables now point to the local xray_aux path instead of FTP in the xray image
- ISIS, ISISscript installed (and SLang)
- NICER estimator tool nibackgen3C50 installed

HETDEX
- The HETDEX compute image has been resfreshed and contains the newest version of the HETDEX API

A list of all installed software can be found in the readme file of the getting_started data volume
https://sciserver.mpe.mpg.de/dashboard/files/datavolumes/4
As always, if are missing any software for your work, let me know and I will install it!

cheers
Jonas 


—
Jonas Haase
Max Planck Institute for Extraterrestrial Physics (MPE)
Giessenbachstr. 1, 85748 Garching, Germany
X5 1.3.08
+49 89 30000 3706