NASA logo

Installation :
6.4. Installing it

[Home] [Prev] [Next] [Up] [Index]    [Subsystems] [Java Subsystems]

If you are reading this, you have probably uncompressed and de-tarred the distribution. Don't do anything further until you read these directions.


[Top] Step 1: Go through the pre-flight checklist.

Please go through the preflight checklist carefully.

In particular, you must build your HTTP server in accordance with the instructions. You should review the pre-flight checklist when installing or reinstalling; the requirements may have changed since your last install.


[Top] Step 2: Unpack distribution in ~imswww.

As user imswww, move the distribution .tar.gz (or .tgz) file to the ~imswww directory, uncompress, and untar it. Like so:

    gzip -d distfile.tar.gz  or
    gunzip distfile.tgz
    tar -xvf distfile.tar

WARNING!All files in the distribution should be owned by ~imswww/. If you are not logged in as ~imswww, do so now, and bring the files over from wherever they currently are.

You should now have a directory structure like this:

  ~imswww/
     |
     |--- imswww-VERSION/
     :     |
           |--- bin/ ...
           |--- cgi-bin/ ...
           |--- docs/ ...
           |--- src/ ...
           |--- lib/ ... 
           :

If this is not the first time you've done this, you may have directories for other versions of EOS Data Gateway at the top level.


[Top] Step 3: Unpack the support files in ~imswww

You need to set the support files if you don't have them from a previous installation.

As user imswww, move the support_files .tar.gz (or .tgz) file to the ~imswww directory, uncompress, and untar it.


WARNING!All files in the support_files directory should be owned by ~imswww/. If you are not logged in as ~imswww, do so now, and bring the files over from wherever they currently are.

You should now have a directory structure like this:

  ~imswww/
     |
     |--- support_files/
     :     |
           |--- sup_files/ ...
           |--- dist_files/ ...
           :

All EOS V0 clients, including the EOS Data Gateway, depend on a directory tree of special supporting files. These files come in two sets. The first set is sometimes referred to as the "distribution files" found in support_file/dist_files/ is a static set of files. The second set of files are sometimes called the "support files" and sometimes called the "valids" and are found in support_files/sup_files.

Be sure that imswww has read and write access to this directory and also to the parent directory of support_files.


[Top] Step 4: ``Configure'' the system.

You are now ready to give EOS Data Gateway the information it needs to compile and link the gateway programs.

Note: If you are building on a Solaris machine you may need to include "/usr/ccs/bin" in your path.

If you have previous version of the web gateway installed on your system you may want to look at the myconfig script used to build that version.

Edit the script called "myconfig", which should look like this:

    #!/bin/sh
    # myconfig -- invoke configure script
    ./configure \
	--with-debug \
	--verbose \
	--with-perl=/usr/local/bin/perl5.6.1 \

As a first cut supply just the Perl path.

Change to the top-level directory of the distribution and invoke the configure utility, like so:

    cd ~/imswww-VERSION

Now run the myconfig script:

     ./myconfig 

This may take some time, as the configure script will try and determine certain characteristics of your system: what C compiler to use, what functions are available, etc. Once it is done, it will go through your distribution and generate certain files needed for the build (primarily Makefiles) from the corresponding *.in (configure INput) files.

If at any point you think that configure has made an undesirable choice, you should interrupt it and restart it, supplying the correct values in the "myconfig" script. Type:

    configure --help

and look for the --with-package options at the end of the listing.


WARNING!If you are re-running configure because it guessed wrong, make sure to delete the "config.cache" file... otherwise, it may use the same cached values.


[Top] Step 5: ``Make'' the system.

From the top-level directory of the distribution, invoke your GNU make program, like so (for clarity, we will refer to it as "make", but you should use whatever command is appropriate, like "gmake"):

    cd ~/imswww-VERSION
    make install >& make_install.log &

This will compile the system, copy the binaries over to their appropriate locations, create necessary symbolic links, etc.

When the make install is done, carefully read through the make_install.log file looking for Error messages. Numerous Warnings are common and can usually be ignored, if an error occurred then the problem must be fixed and make install rerun.


NOTE!There will be a warning during the installation of libwww-perl which implies that an FTP module was needed but not found. Since the EDG doesn't access any ftp://... links internally, it doesn't need this module and this isn't a problem


[Top] Step 6: Create symbolic links.

The following links make the installation visible to the web via the ~imswww/public_html directory. You need to verify that user public html directories on your system are called public_html before proceeding with this step. If the user public html are called something different then use that name instead of public_html.

In the home directory of "imswww" create the following directory structure for the HTTPD to access if they don't already exist from a previous installation.

      cd ~imswww
      mkdir public_html
      cd ~imswww/public_html
      mkdir cgi-bin
      cd  ~imswww/public_html/cgi-bin

Create links for this installation:

    cd ~imswww
    ln -s imswww-VERSION VERSION
    (For example: ln -s imswww-1.5 1.5)
    cd ~imswww/public_html:
    ln -s ~imswww/imswww-VERSION/docs VERSION 
    (For example:  ln -s ~imswww/imswww-1.5/docs 1.5 ) 
    cd ~imswww/public_html/cgi-bin:
    ln -s ~imswww/imswww-VERSION/cgi-bin VERSION
    (For example: ln -s ~imswww/imswww-1.5/cgi-bin 1.5)


[Top] Step 7: Generate and apply the SETUP file.

You are now ready to give EOS Data Gateway the information it needs to install and run the gateway. From the top-level directory of the distribution, check the setup utility. Edit this file and make sure that the Perl path on the second line is correct for your site. Then, invoke the setup utility, like so:

    ./setup -make

This will create a file called SETUP with default values. Edit this file to make certain it contains the correct information for your site.


[Top] Required:

At a minimum you need to change the following lines to the appropriate values for your site (full descriptions of all the values are described elsewhere in this document):

AdminCrypt
The crypted password for the Gateway administrators to use, exactly as you'd see it in an /etc/passwd file. This password will be provided by the EOS Data Gateway administrators. For example:
       AdminCrypt     "encrypted-pw-provided"

WARNING!The special value "NO PASSWORD" allows ANYONE in!!! PLEASE, DO NOT USE AN ACTUAL SYSTEM PASSWORD FOR THIS!

CgiUrl
Set this to "/ims-bin/VERSION/". For example:
       CgiUrl         "/ims-bin/1.5/"

DocsUrl
Set this to "/~imswww/VERSION/". For example:
       DocsUrl        "/~imswww/1.5/"

EcsProfileSmc
Set this to "SMC-UPG". For example:
      EcsProfileSmc   "SMC-UPG" 

GaeaDataDir
Set this to the full path to your valids. This MUST match the directory you created in Step 3. For example:
       GaeaDataDir "/home/imswww/support_files/sup_files"

GaeaDistribDir
Set this to the full path to your distribution support files. This MUST match the directory you created in Step 3.
       GaeaDistribDir "/home/imswww/support_files/dist_files"

MorePaths
Be sure it includes the location of your wais software. For example:
       MorePaths   "/usr/bsd:/usr/local/bin:/usr/local/wais/bin"

PerlHttpdPort
Choose a port on which to run the Perl httpd. is recommended that you run the Perl httpd. Running the Perl httpd will significately speed up the performance of the system, by eliminating the need to reload the Perl libraries for each CGI process. At the current time, we do not recommend running with PerlHttpdPort set, because we have been having some problems with it.
	PerlHttpdPort "8123"

We recommend setting it to:

        PerlHttpdPort ""

WARNING!Port numbers are very site specific. Check with your system administrator to get an appropriate port number.

HomeEmail
Set the e-mail address to where comments are sent. This should be set to the Data Center User services group where the machine is hosted.

For example:

    HomeEmail  "edc@eos.nasa.gov"

HttpdHost
This is the hostname that will be used in URLs pointing back into the system, and also for email sent to/from this site; leave it empty ("") for the default hostname.

For example:

    HttpdHost "harp.gsfc.nasa.gov"

MaxGransDefault
This is the number of granules returned per dataset when a user issues a query.

For example:

    MaxGransDefault  "100"

UseNonSpatial
This is a switch (Y or N) used if this installation supports the special ODL flag for including non-spatial granules in the results. If set to Y, a checkbox option will be displayed on the search form within the Choose Additional Options, ALL Options section. The default value is empty string or N.

For example:

    UseNonSpatial  "Y"

UseNonTemporal
This is a switch (Y or N) used if this installation supports the special ODL flag for including non-temporal granules in the results. If set to Y, a checkbox option will be displayed on the search form within the Choose Additional Options, ALL Options section. The default value is empty string or N.

For example:

    UseNonTemporal "Y"

SiteAdmin
Set the e-mail address to where system error messages ("dying gasps") and other information is sent.

For example: SiteAdmin "imswww"

SiteName
Set the name you wish to use for your installation.

For example:

    SiteName  "Goddard Space Flight Center"


[Top] (Optional) Site Administration :

DisableByDiscipline
For special case installations of the EDG, non-standard valids may not have all the information needed to drive the By Discipline feature on the search form. For those installations you can disable the By Discipline feature by setting this flag to "YES". Leave blank to enable By Discipline.

For example:

    DisableByDiscipline "YES" 

SiteSpecDir
Specify the directory you will use to hold site specific info and documents for the gateway, such as welcome_header.html.

For example: SiteSpecDir "/usr/people/imswww/site_specific"

PolicySecurity
Normally set to "US Government" - this will cause a link to be created on the welcome page to the US Government security policy. This is required for all US installations.

However installations at non-US sites may want to substitute their own security policies. In which case, set this to "local". If you don't want any security policy link on the welcome page then set it to blank "".

PolicyPrivacy
Normally set to "US Government" - this will cause a link to be created on the welcome page to the US Government privacy policy. This is required for all US installations.

However installations at non-US sites may want to substitute their own privacy policies. In which case, set this to "local". If you don't want any privacy policy link on the welcome page then set it to blank "".


[Top] XML Guide Search (Spidex) Configuration

You will need to check the Guide Configuration Page, located at this URL.

This page specifies the values you'll need to enter for SpidexUrl and SpidexConf.


[Top] (Optional) Persistent User Accounts:

Beginning with release 2.0, the EOS Data Gateway can be set up so that user accounts can be established. These accounts permit a user's session to be active for an extended period of time.

To enable User Accounts (Persistent User Sessions):

HavePersistentSessions
Allow the users to have Persistent directories on the server. The default is "NO", change it to "YES" to enable this option.

PersistentDir
Base location of the Persistent directories , for instance :
	"/imsdev/persistentUser/"

Please don't have this directory "tied" to a particular version of the Gateway. We recommand the directory reside under ~imswww/.

In most cases, the Web Gateway's defaults can be used for the following :

AccountMax
How much disk space should be allowed for each user account. This maximum limit should be enough to handle the results of a large search. If a user exceeds this size and they are not signed-in, their results will be purged. It they are signed-in they will be warned. Specify a size with units of Mega Bytes: e.g., "20mb". Recommend: "20mb".

For example: AccountMax "20mb"

AllAccountsMax
How much disk space should be allowed for ALL user accounts. This maximum limit should be quite large, since each account can be typically 1-5 MB (although could be smaller or bigger). Specify a size with units of Mega Bytes or Giga Bytes: e.g., "500mb" or "2gb". Recommend: "1gb".

For example: AllAccountsMax "1gb"

CriticalAllAccountsMax
How much disk space should be allowed for ALL user accounts before more drastic measures are invoked to clear space. This maximum limit should be quite large, since each account can be typically 1-5 MB (although could be smaller or bigger). It should generally be larger than AllAccountsMax. Specify a size with units of Mega Bytes or Giga Bytes: e.g., "500mb" or "2gb". Recommend: "2.5gb".

For example: CriticalAllAccountsMax "10gb"


WARNING!Again, accounts sizes are very site specific. The size of the accounts depends on capabilities of the system as well as the usage of the EOS Data Gateway.

GuideKillTime
Under CRITICAL disk space conditions (account usage exceedingCriticalAllAccountsMax), if guide search results are not accessed for this period of time, then they will be deleted - even from accounts where the user is signed in. Specify a time, optional spaces, and units: e.g. "5m" for 5 minutes. Recommend: "15 m".

For example: GuideKillTime "15m"

LogoutTime
How long should a logged in account be inactive before we log the user out? Used by imscleanaccounts.pl. Specify integer time, optional blanks, and units. Recommend: "1 d" for one day.

For example: LogoutTime "1 d"

ResultsLifetime
How long can results be kept in a user account, that is not accessed, before they are deleted (to free up disk space and keep info from becoming out-of-date)? Used by imscleanaccounts.pl. Specify integer time, optional blanks, and units: e.g., "10d" for 10 days. Recommend: "14 d" for fourteen days, the valids cycle.

For example: ResultsLifetime "14 d"

SecuredDir
Location of the secured subdirectory of cgi-bin (for persistent logins). Defaults to "secured".

For example: SecuredDir "secured"


[Top] (Optional) Detailed Map:

If you want to enable the more detailed map for the map selection applet, you must follow the following directions.

MapCgiScript
This is the map server script used by the Geo selection java map. By default this is set to "imswdbtile.cgi" which is a low resolution map with maximum zoom of 16x (state level) and has the smallest map database.

For Example:

 MapCgiScript "imswdbtile.cgi"

Set to "drawglobe.cgi" for higher resolution maps and more features.

For Example:

 MapCgiScript "drawglobe.cgi"

For this option you have a choice of installing one of three resolution levels. Pick the resolution level by setting the MapResolution.

Possible values for MapResolution are:

MEDIUM
Uses the lowres and medres databases (25MB total) and allows zooming to 64x (cities and major roads).

HIGH
Uses the lowres, medres and highres databases (237MB total) and allows zooming to 500x (towns and secondary roads).

ULTRA
Uses the lowres, medres, highres, and ultrares databases (1331MB total) and allows zooming to 1000x (neighborhoods and residential roads).

Here are the directions for installing the map data bases:

  1. Read the SETUP file and find the location of the support_files directory as indicated by the setup variable GaeaDistribDir.

  2. Switch to the directory of GaeaDistribDir.

    For example: cd ~imswww/support_files/dist_files

  3. ftp lyta.gsfc.nasa.gov

    Name: anonymous

    Password: your-email-address

  4. cd outgoing/edg

  5. For all resolutions:

    get map_symbols.tar.gz

    get map.tar.gz

  6. For HIGH and ULTRA resolution maps:

    get highres.tar.gz

  7. For ULTRA resolution maps:

    get ultrares.tar.gz

  8. quit

  9. Use "gunzip" or "gzip -d" to uncompress the files:

    gunzip map.tar.gz

    gunzip map_symbols.tar.gz

    gunzip highres.tar.gz

    gunzip ultrares.tar.gz

  10. Untar the files:

    tar xvf map.tar

    tar xvf map_symbols.tar

    Untar highres and ultra database, if needed, in the map directory:

    cd map

    tar xvf ../highres.tar

    tar xvf ../ultrares.tar


[Top] Apply the SETUP file:

When you are done editing the SETUP file, save it. Then, apply the contents of SETUP by running:

    ./setup -apply


[Top] Step 8: Transfer the valids.

Updates to the valids are provided by the EOS Data Gateway Team every two weeks. You obtain the latest set of valids by executing the "run_autoxfer.pm" script supplied with the web gateway. Run it now to get the valids:

    ~imswww/imswww-VERSION/lib/perl/run_autoxfer.pm  force

In the step 11 you will set this script up to run each night in a cron job, to automatically keep your valids up to date.

The command line argument 'force' is new with version 2.0 of the EOS Data Gateway. It should be used ONLY during the installation run and NOT in the cron job.


[Top] Step 9: Make valids file for e-mail ordering.

The wwwvalids.conf file is used by the system for e-mail ordering. Which data centers and data sets available of e-mail ordering along with the e-mail address to which send orders are stored here. The editable text version must be converted into a gdb file for use by the system. The gdb version of the file must be recreated on each system as gdb files have platform dependences in them.

    cd ~imswww/imswww-VERSION/lib/conf
    cp ~imswww/support_files/sup_files/wwwvalids.conf ./
    ../../bin/mkvalids

If wwwuser is set up as a group that is shared with imswww you should do the following:

    chgrp wwwuser wwwvalids.gdb
    chgrp wwwuser wwwvalids.gdb..LCK
    chmod 664 wwwvalids.gdb
    chmod 664 wwwvalids.gdb..LCK
    chgrp wwwuser misrData.gdb
    chgrp wwwuser misrData.gdb..LCK
    chmod 664 misrData.gdb
    chmod 664 misrData.gdb..LCK
    cd ../..
    chgrp wwwuser log
    chmod 775 log
    chmod 755 src/sf_client/autoxfer
    cd tmp
    chgrp wwwuser imswww-usr
    chmod 775 imswww-usr

If wwwuser is not set up this way you must:

    chmod 666 lib/conf/wwwvalids.gdb
    chmod 666 lib/conf/wwwvalids.gdb..LCK


[Top] Step 10 (Optional): Make site specific directory and files.

To tailor the gateway welcome screen to your site, we have provided a mechanism for you to do this.

First, create a site specific directory if you don't already have one. It must have the same name specified in SiteSpecDir in the SETUP file from Step 7 above.

    mkdir ~imswww/site_specific

You can now create a file in this directory for any text, links, graphics, or html compatible information that you want to appear at the top of the Welcome page. The file MUST be called:

    welcome_header.html

You can now create a file in this directory for any text, links, graphics, or html compatible information that you want to appear at the bottom of the Welcome page. The file MUST be called:

    welcome_footer.html

If you set one or both of PolicySecurity or PolicyPrivacy to "local" in your SETUP file, then you need to create a local security policy file and/or local privacy policy file in your site specific directory.

For PolicyPrivacy = "local" the file Must be called:

    input_privacy.html

For PolicySecurity = "local" the file Must be called:

    input_security.html


[Top] Step 11: Set up crontab for imsstats, imscleanup, autoxfer, makewelcome, and (optionally) imscleanaccounts.

The wwwuser and imswww accounts should have cron jobs running to handle periodic execution of some support scripts. Check your unix documentation for the proper use of crontab.

Crontab for wwwuser

All actions (events) are logged by the EOS Data Gateway software. We would appreciate if you could send us a copy of your EOS Data Gateway logs. NASA is very interested in knowing how many people are using the system and what features are being used most often. The information in the logs is important to us in helping us build better future EOS Data Gateways. Simply add an entry for the program imsstats to the crontab for the ~wwwuser account to be run once a day.

The EOS Data Gateway has an imscleanup script which should be run periodically (about every 15 minutes) to clean up cache files and stale user sessions. The most reliable way to invoke it, by far, is as a cron job.

(Optional) If your site is set up to have persistent user accounts, then a separate cleanup script, imscleanaccounts.pl, should be run periodically, about once per hour. Again, it is easiest to do this in a cron job.

To setup the three scripts to run periodically, login as user "wwwuser" and add the imsstats and imscleanup routines to the crontab. For example:

   # Once a day mail EOS Data Gateway logs to the imswww-stat account at GSFC:
   0 1 * * * /usr/people/imswww/pub/bin/imsstats -cron >/dev/null
   # Cleanup the EOS Data Gateway's session files -- 2.0 area:
   0,15,30,45 * * * * /home/imswww/2.0/lib/bin/imscleanup -cron >/dev/null
   # Cleanup the EOS Data Gateway's session files -- 2.1 area:
   1,16,31,46 * * * * /home/imswww/2.1/lib/bin/imscleanup -cron >/dev/null

Please note that you need to have an entry in the crontab to run imscleanup for each of your installations of the web gateway. As versions of the web gateway are installed or deleted you need to add or remove the appropriate crontab entry. You don't need to cleanup installations that are not being used. Only one entry is needed for imstats as we are only interested in the log files from whatever is the current public installation.

If you have persistent user accounts in your EOS Data Gateway, then the cron job should run imscleanaccounts once per hour. You only need one of these for the persistent user accounts, even if you have multiple installations. The entry in the crontab should look like this:

   # Once per hour cleanup persistent user accounts.
   2 * * * * /home/imswww/pub/lib/bin/imscleanaccounts.pl -cron >/dev/null
   10 3 * * * /usr/people/imswww/pub/bin/buildBridge >/dev/null

Crontab for imswww

The welcome pages contain a "What's New" section which needs to be updated periodically. To do this, you need to rerun the makewelcome script. This script runs quickly, accessing a Web page from another server and copying part of its text into the welcome page. We recommend that this be done daily; updates are not that frequent, but it is more convenient to run this script on a daily basis.

To set up the makewelcome.pm program, log in as user "imswww" and add the routine to the crontab. For example:

    # Get the current "What's New" entries:
    0 4 * * * /usr/people/imswww/pub/lib/perl/makewelcome.pm >/dev/null

The "valids" you obtained in step 8 are updated by the EOS Data Gateway Team every two weeks. To be sure to keep your valids updated, the run_autoxfer.pm script should be run periodically. This script runs very quickly. It can take less than one second if no new valids are available. Therefore, we recommed that you simply add an entry for the script run_autoxfer.pm to the crontab for the ~imswww account to be run once a day.

To setup the run_autoxfer.pm program, login as user "imswww" and add the routine to the crontab. For example:

   # Get the latest valids for the public installation:
   0 3 * * * /usr/people/imswww/pub/lib/perl/run_autoxfer.pm -cron >/dev/null

NOTE:

The cronjob program run_autoxfer copies the wwwvalids.conf file included in the valids distribution into the EDG installation. However one set of support_files and therefore one run_autoxfer program often supports multiple EDG installations, this means that wwwvalids.conf only gets updated in the EDG installation which actually ran the transfer. Other installations which also use the same support_files are not updated with the new wwwvalids.conf. In the event that an installation is sharing the valids with another installation, transfer the latest wwwvalids.conf file using the following command:

   autowwwvalids -l supportFileDir -t topEDGInstDir

Where "autowwwvalids" is the script that performs the transfer. It should be located in the /lib/bin directory of the installation.

"-l" and "-t" are command arguments needed by the script.

"supportFileDir" is the absolute path of the support file directory.

"topEDGInstDir" is the absolute path of the top level directory for the installation where the wwwvalids.con file will be copied.

For example:

   # Once a day check/transfer the latest wwwvalids.conf file
   0 2 * * * /usr/people/imswww/pub/lib/bin/autowwwvalids -l /usr/people/imswww/support_files/sup_files -t /usr/people/imswww/pub >/dev/null


[Top] Step 12 (Optional): Starting the custom httpd Perl script server.


WARNING!We do not recommend running with the custom httpd Perl script server at this time.

If you did not set the variable PerlHttpdPort in the setup step and you wish to run the custom httpd go back to Step 7: Generate and apply the SETUP file. and do so. If you did not set the variable PerlHttpdPort and do not want to run the custom httpd you need not do anything.

To reduce the time it takes to run CGI Perl scripts, a custom httpd server imshttpd is run. The Perl server has the the Perl modules preloaded, so when a user requests a CGI-generated page in the EOS Data Gateway the load time for the Perl is reduced. Non-CGI pages are served via your systems standard httpd server.

A imshttpd is run for each installation of the EOS Data Gateway on the system. Different versions of the EOS Data Gateway do not and can not use the same imshttpd server. Each server needs its own port number (which was set in the SETUP step).

Each version has its own imshttpd:

     imswww-2.3 on port 8126,
     imswww-2.4 on port 8127.

WARNING!Port numbers are very site specific, we can not tell you what would be an appropriate port number to use. Port 8123 maybe used by other software on your system. Check with your system administrator to get an appropriate port number.

To start the imshttpd, login to the account "wwwuser". and execute:

     ~imswww/VERSION/lib/bin/imshttpd

A successfully started imshttpd will end with the message:

     Bottom line: OPERATION SUCCEEDED!

You will want to have your system administrator add the starting of the imshttpd's to the system init file so that they will be automatically restarted when the system is rebooted.


[Top] Step 13: Try out the installation

Get into your favorite WWW browser, and open the URL:

    http://<your_host_name_here>/~imswww/VERSION/imswelcome

For example:

	http://harp.gsfc.nasa.gov/~imswww/1.5/imswelcome

You should see the ``welcome'' page.


[Home] [Prev] [Next] [Up] [Index]    [Subsystems] [Java Subsystems]