PAM Windows Workstation

Note

All workstations will be restarted on the 15th of every month at 22:00 UTC for required patching and updates. Please plan your processing accordingly.Each pam-ww can be turned on/off here.

Accessing or updating PAM windows workstation

The pam-ww are provisioned to each individual interested in using them for cloud processing or data validation. To request a pam-ww instance, submit a new issue here and select the “Request a pam-ww” template. If the previous link does not work for you, please submit this form instead. The cloud team will be notified of your request from either form and will comment on the issue or email you when the pam-ww is ready for use.

Required setup before accessing pam–ww for the first time

Configure gcloud

All steps in this section only need to be done once for each computer you want to access the pam-ww from.

  1. To connect to a remote workstation, you have to install software on your primary physical computer. The next steps in this section assume you’re using Windows, alternatives exist for Mac and Linux.

  2. Install Google Cloud SDK on your primary physical computer. In the section titled “Installing the latest gcloud CLI version”, select your personal computer operating system, click “Google Cloud CLI Installer” and follow the steps to configure the client software. See the image below for comparison. Once Cloud SDK is installed, a gcloud terminal window might open. You can close this terminal.

  1. Open the Windows Command Prompt (aka “terminal”) and enter “gcloud auth login”. This will take you to a browser to login with your email password or two-step verification to configure your user credential with gcloud. This step can completed using shells on other operating systems as long as they work with gcloud.

  2. If prompted to set a project, enter the following code chunk:

    gcloud config set project ggn-nmfs-pamdata-prod-1

  3. To confirm authentication was successful, enter ‘gcloud auth list’ in the terminal and check that it returns your NOAA email address.

  4. Once you have confirmed the gcloud auth login was successful, you can close the Windows terminal and move on to the connect steps to access your pam-ww.

Step-by-step access instructions

Turn pam-ww instance on


Prior to connecting, you need to turn your pam-ww instance ‘on’. You can navigate to the link here to see the workstation state and turn it on/off in the browser.

Green checkmarks means on, circle means state is changing (turning on or off), gray means stopped. The three dots on the right are called the ‘hamburger’ menu where you can turn change the on/off state.

Find the instance containing your name, click on the three vertical dots on the right of the row representing that instance, and select ‘Start/Resume’. When you are done working turn your pam-ww off by returning to this page, selecting the hambuger menu, and clicking ‘Stop’. This is also how you turn it off.

Note

Please keep the instance off when not in use, for instance at the end of your workday if you are actively using it. Leaving it on for long periods unnecessarily incurs unneeded cost.

Remote Desktop Connection


Once your pam-ww instance is on you can proceed to login by submitting a single line of code through the Windows Command Prompt (terminal). This code will access the pam-ww in the cloud then use the built-in windows remote desktop client that is usually preinstalled. Step-by-step instructions are:

  1. Open up the Windows Command Prompt if on windows.
  2. Modify the below line of code to change the “firstname-lastname-pam-ww” to match the name of your workstation. Then enter the whole chunk of code into the command line to log in as a user. You can reference the name of your workstation on the pam-ww management page.
start gcloud compute start-iap-tunnel firstname-lastname-pam-ww 3389 --local-host-port=localhost:3390 --zone=us-east4-c --project=ggn-nmfs-pamdata-prod-1 && timeout 8 && mstsc /v:localhost:3390

For example, for user “rebecca.vanhoeck@noaa.gov”, you would enter in command prompt:

start gcloud compute start-iap-tunnel rebecca-vanhoeck-pam-ww 3389 --local-host-port=localhost:3390 --zone=us-east4-c --project=ggn-nmfs-pamdata-prod-1 && timeout 8 && mstsc /v:localhost:3390

  1. After entering the code a second terminal window will open and state that it is attempting to connect. If successful, the RDP client will open and prompt you to log in as a user. On this RDP pop-up window, select pam_user for user, do not supply a password, and click ‘OK’. The remote desktop connection will supply the correct username on subsequent logins.

    Note

    • If a Windows firewall pop-up appears and asks for admin login, select cancel on the pop-up and move on.

    • If the RDP is asking for an authentication method other than a password or is trying to connect to your NOAA account, click “More choices”, then select “Use a different account”, type “pam_user” as the username, do not supply a password and click ‘OK’. If ’pam_user is already an account option, select that one and continue.

    • If the RDP client provides and error that the pam-ww cannot be accessed close the RDP and terminal windows, confirm your instance has a green check mark on this page, then return to Step 1 of the Remote Desktop Connection and try again.

  2. The pam-ww will then open like any remote desktop connection. As the pam-ww opens, accept any warnings that look reasonable. If the code was unsuccessful, ensure your workstation is on and showing the green checkmark, then return to step 2 and try submitting the code again.

  3. Leave the terminal windows open while working on the pam-ww. If you submit a long processing job you can close the remote desktop window, but leave the pam-ww on to allow the job to continue.

Note

Some users report issues associated with connecting with the above command. If you have similar issues, try running in cmd with:

gcloud compute start-iap-tunnel rebecca-vanhoeck-pam-ww 3389 –local-host-port=localhost:3390 –zone=us-east4-c –project=ggn-nmfs-pamdata-prod-1

Like above, change the above code to match your username. Then, leaving the terminal window running, navigate to “remote desktop connection” application, and enter in localhost:3390. Like with above, supply the username as pam_user and no password.

For even quicker logins, you can copy the code into a text file, rename to make it have a .bat extension instead of .txt extension (if your primary physical computer is Windows), and then double click it. The code will run in the cmd line when you double click it, so you don’t have to open cmd prompt and copy paste the code in each time.

Configuring user mounts


NMFS passive acoustic data in the cloud is available through the pam-ww with broad read level access within the desktop shortcut “Mounted_Acoustic_Data”.

To improve connection to the data buckets from some software programs or to conduct data management tasks from the pam-ww, data buckets can be mounted to a letter drive through “user mounts”. In this use case, users are able to apply their full set of write-access permissions to perform administrative tasks on passive acoustic raw data or write to an FMC specific data bucket for analysis. For example, those with data admin permissions will be able to rename, move, or delete audio data from their FMC data bucket.

User mounts are mounted to certain letter drives by default, and each user is free to specify which drive and buckets they would like to mount with their own user permissions. Note that this capability will not grant your user permissions that were not previously granted by cloud admins: ensure you already have the needed permissions for proper functioning.

The steps to configure user mounts are as follows.

  1. Open up the folder called ‘user_drive_mounts_config’ on the desktop.

  1. Open up the files my_mounts.txt and my_mounts_template.txt

  1. Use the syntax specified in my_mounts_template ([drive_letter]:[gcp_bucket_name] …) to specify your desired mounts in my_mounts.txt. For instance, if you were an NEFSC user, you might choose to specify:
N:nefsc-1 M:nefsc-1-detector-outputs

Write your desired drive bucket mounts and then save the my_mounts.txt file.

  1. To get the mounts running, you can click on ‘start_user_mounts.bat’ in the same folder.

  1. The pam-ww will attempt to reuse your authentication as long as it is valid. The first time it runs, and when Google needs to refresh your authentication, the pam-ww will prompt you to login and give user consent with Google. When this happens, enter ‘Y’ in the command window that the ‘start_user_mounts.bat’ opens to complete the browser login and user consent flow.

Ensure you check any boxes provided to you from within the browser process for the authentication to complete successfully.

  1. If successful, you will see a success message in the cmd window. You will also see in Windows Explorer that new drive mounts are now available. Keep this window running as long as you would like the user mounts to be running, and if you accidentally close, you can run the ‘start_user_mounts.bat’ file again to launch the user mounts.

  1. If your needs for user mounts change, you can just go back to the my_mounts.txt file and specify a different set (or no) mounts as desired. Like before, these changes will take effect on login, or when you next run the ‘start_user_mounts.bat’ file.

  1. Unlike the read mounts, the drive mounts substitute the entire name of the bucket for the specified letter. Meaning, if I wanted to access ‘mooring1’ in NEFSC-1/bottom mounted, and the NEFSC-1 bucket was mounted as the N drive, in my application I would path to N:/bottom_mounted/mooring1

Mounts can safely be closed at any time by exiting the running cmd window, logging off, or shutting down the workstation. To re-initate your mounts, run the ‘start_user_mounts.bat’ file.

Streamlining pam-ww access

To more easily open your pam-ww without having to enter code into the command prompt, you can save the access code as a .bat file. Then double click that file to be taken to the Remote Desktop Client to log in.

  1. Copy the below line of code in to a notepad document.
start gcloud compute start-iap-tunnel firstname-lastname-pam-ww 3389 --local-host-port=localhost:3390 --zone=us-east4-c --project=ggn-nmfs-pamdata-prod-1 && timeout 8 && mstsc /v:localhost:3390}
  1. Modify the code to change the “firstname-lastname-pam-ww” to match your name and the name of your workstation.
  2. Save this note as a .bat file by navigating to “File”, then “Save As”. In the “Save as type” dropdown select “All Files”, then name the file something easily identifiable inlcuding the .bat extention in the file name (e.g. “pam_ww_startup.bat”). This will be successful if the file appears in the location you saved it with the file type “Windows Batch File”.

Once the .bat file is created, you can access your pam-ww by turning it on and then double-clicking the .bat file. This will take you to the Remote desktop client for “pam_user” select “OK” and start up should proceed as normal. If opening the file doesn’t work on the first try, simply close the error box and try again. It typically works the second time.

Note

Important: Please remember to turn your pam-ww instance off when it is not actively completing a job. Leaving it on for long periods unnecessarily incurs cost. To turn your instance off, navigate to the link here, click the three vertical dots by your instance and select “Stop”.

Specific software guidance

Anaconda

Anaconda is licensed software that we are not allowed by Terms of Service to redistribute on the pam-ww. Additionally, users need or prefer various flavors of Anaconda, such as GUI anaconda, miniconda, etc. As such it is simpler for users to self install the version of Anaconda they need, and users are able to perform this on their own without admin permissions.

  1. Ensure you are on the latest pam-ww version, and if not, consider an upgrade. This is not mandatory, but is a good idea tp get this out of the way since your Anaconda install will not persist in a newly provisioned pam-ww.
  2. Log in to your pam-ww and open Google Chrome. Navigate to https://www.anaconda.com/download (CLI), or https://www.anaconda.com/products/navigator (GUI), or the appropriate download page for any other Anaconda product as desired. Provide your personal email to access the installer download. Once downloaded, double click and go through the install steps, ensuring you select options to install for a single user and not on the system to avoid the prompt for admin rights.
MATLAB

MATLAB is licensed software that we are not allowed by Terms of Service to redistribute on the pam-ww. It necessitates admin rights, so you will need an admin to assist in it’s install. Steps 1-4 below can be completed independently, but step 5 will require coordination with Dan Woodrich. Please submit a Gitub issue to this repo to schedule a time to meet with Dan.

  1. Ensure you are on the latest pam-ww version by requesting an upgrade if not currently on the latest version. This is mandatory, as we want to minimize the # of MATLAB support sessions as much as possible.
  2. Log in to your pam-ww and open Google Chrome. Navigate to mathworks.com and sign in to access the license center.

  1. Go to your account, select a license, and then go to the “install and activate” tab.

Mathworks allows for you to use a single license on two different computers. If necessary, you can deactivate a computer here to free up a license for use in the cloud.

  1. Select the product you’d like to download. If there are multiple download options, select the first one to “Download and run the installer”, not the the Update. The installer will download to your downloads folder. This is the last step you can complete without having a temporary admin password.

  1. For this step you will need to have an admin on hand, so please create a github issue in this repo or email Dan Woodrich to schedule a session. Admin will stand by during this step to provide you an admin password that will be changed immediately after your use. A video call is required as the admin will need to ensure you are using the temporary credential appropriately.

Once online with the admin, open the installer. Provide the requested fields. The installation process varies slightly for each version of Matlab. A few possibility are shown below. If prompted for the username that will be associated with the license, ensure you specify that pam_user is the Computer login name.

Example from R2023b. If prompted for the computer host id, this can be found by opening up a cmd prompt and running ‘vol’.

  1. After MATLAB has been installed, admin may still be desirable to set search paths that will persist between sessions. The admin can again provide you a temporary admin password for this purpose.

Resolving issues and requesting updates

If you are encountering challenges successfully operating on the pam-ww there are a few troubleshooting steps and ways to request support. In general, a troubleshooting workflow should include (1) reviewing the technical advice listed in the next section to see if there are documented suggestions for your use case; (2) checking which version of the pam-ww you are currently using; (3) Comparing that version with the version history to see if there are updates that might solve your issue; (4) If there is a more recent version, then request an updated pam-ww; and lastly (5) if you are still having issues then submit a bug report or new feature request. Full instructions for each of these steps are below. 

Check pam-ww version and compare to version history

The pam-ww are actively under development as software are tested and bugs are identified and fixed. This means that you may occasionally need to upgrade your pam-ww instance. You can check the current version of your pam-ww by going to the homepage, clicking on the name of your pam-ww, and reviewing ‘Boot disk storage image’ listed in the Basic Information section.

The text under the ‘image’ field indicates the current version of your pam-ww. The pam-ww image name is between the text “pww-disa” and “hardened…”, in this case, “beta-3”. To check whether there are more recent pam-ww versions and identify if the bug you are experiencing has been resolved in one of those versions, can cross reference your current version with pam-ww template change tracking document linked below. Both documents track the changes and are organized in consecutive order where the last entry represents the most recent available version.

Request an upgraded pam-ww

The pam-ww is upgraded through versioning to fix bugs and add new features and software. Currently the default support model is just to replace pam-ww when upgrades are needed. Prior to upgrading your pam-ww:

  1. Ensure all processing jobs on your current pam-ww are complete

  2. Confirm any local files saved on the C: drive of your pam-ww either can be deleted or have been moved to a permanent location. This can either be done by uploading them to google drive, emailing them to yourself, moving them to a detector-output bucket, or moving them to your own folder in the pam-ww-tmp bucket. Note that the pam-ww-tmp bucket is intended to be for temporary storage. Any files there should be regularly deleted or downloaded to your local computer for permanent storage.

  3. Lastly, delete your current pam-ww instance by going to the homepage, clicking the three vertical dots associated with your instance and select ‘Delete’.

To request an upgraded pam-ww submit a new issue here. Click the green “New Issue” button, then select the “Request an upgraded pam-ww” template. If the previous link gives you an error, please use this form to request an upgrade instead. If you had any customized software installed on your instance that you will continue to need on the new pam-ww (eg. Matlab), be sure to include the details of that software in your request. The cloud team will be automatically notified, follow up with any questions, and begin working on your request.

See the next section on requesting new features if there are specific software you’d like installed on your workstation.

Report a bug or request a new feature

Use this form to report a processing issue or request a new feature on the pam-ww. Our cloud development team will get an automated email with your request and will start working on it soon. To check in on the status of your bug/feature request view this smartsheet tracker here.

If there are specific software you’d like installed on your workstation, the cloud team will remote into your instance and install the software on your behalf. In some cases software needs admin privileges as well as personal license credentials with the service (such as MATLAB), in these cases we will set up a shared session to install together.

Technical advice for data processing on the pam-ww

  • Issue: Software is taking a long time or having trouble reading or writing files from the cloud data buckets. Windows based software often have difficulty reading and writing to cloud buckets. By mounting the cloud buckets to a letter drive the software is able to connect to the data in a way that it more easily understood within the Windows OS. Version 10 and on of the pam-ww have implemented user mount configurations to improve reading and writing to the cloud buckets. See the “Configuring user mounts” section above. To manually mount a bucket, open the command terminal within the pam-ww and modify the following code chunk to match the bucket you want to mount and the letter drive you want to mount to. For example, this code chunk mounts the nefsc-1-detector-output bucket to the Z: drive. Leave this terminal window open while your job is processing.
C:\Windows\rclone-v1.68.1-windows-amd64\rclone mount pamdata-gcs:nefsc-1-detector-output Z: -o UserName=pam_user --vfs-cache-mode writes --file-perms 0777 --dir-perms 0777 --network-mode --config=C:\Windows\pamdata\rclone.conf
  • Issue: A multi-day processing job was interrupted. The pam-ww are scheduled to automatically reboot at 22:00 UTC on the 15th of every month for mandotory patching. This will interrupt any processing jobs currently running. Please plan your work accordingly so that jobs will finish before the 15th.

Demo Instructions

Raven

  1. Open Raven Pro 1.6 by selecting the shortcut on the desktop. Either provide a license or use the demo version, then interact with Raven in the same way you would locally. 

  2. Choose data of interest that you want to explore or share

    1. On the desktop, open the “Mounted_Acoustic_Data” file explorer shortcut. 

    2. Explore the data buckets and select data of interest. 

    3. Drag and drop sound files into Raven. Select “Page Sound”, then “Open”

    4. If your group doesn’t have data in the cloud, use one of these recommended deployments: 

      1. Cool humpback song: afsc-1/bottom_mounted/GA19_AU_BT01/12_2019

        • Select files: 191201_040000 - 191201_101000

      2. NPRW gunshot song: afsc-1/bottom_mounted/BS12_AU_PM02-a/08_2012

        • Select files: 120805_032000 - 120805_081000

      3. Very loud humpback: nefsc-1/bottom_mounted/NEFSC_GOM/NEFSC_GOM_202309_USTR02/6556_48kHz_UTC/

        1. Select file: 6556.231112050000

      4. Sperm whales: nefsc-1/bottom_mounted/NEFSC_MA-RI/NEFSC_MA-RI_202205_NS05/NEFSC_MA-RI_202205_NS05_ST/6086_64kHz_UTC/

        1. Select files: 6086.221104023559

  3. Explore acoustic data in Raven: Listen to the sounds, explore changing the spectrogram views, making selections, etc. 

  4. Show and Tell: take turns sharing your screen, talking about the data you chose, and what you’ve found! As you share, others are encouraged to follow along and listen/analyze themselves or explore nearby sections or similar times in other nearby sites. Have fun exploring and talking about interesting acoustic data!

R Studio & PAMscapes


Access required files

  1. Create a personal folder to store temporary files
    • Navigate to the GCP Console, Open the “pam-ww-tmp” bucket, select “Create Folder”, and name it with your First_LastName
  2. In the pam-ww, open google chrome and login to your google drive
    • Open this folder and download the available R script to your personal folder in the pam-ww-tmp bucket that is mounted to the X: Drive
    • The file might automatically save in the downloads folder. If so, open the windows file explorer, copy the file from downloads and paste it in your personal folder in pam-ww-tmp.

Mount Data Buckets for faster access

  1. Mount data bucket of interest
    • Open the Windows command prompt (terminal)
    • Copy the code chunk below and enter it into the terminal to mount the bucket that the soundscape metric data are stored in. This code chunk mounts the nefsc-1 data bucket to the Z: drive.
C:\Windows\rclone-v1.68.1-windows-amd64\rclone mount pamdata-gcs:nefsc-1 Z: -o UserName=pam_user --vfs-cache-mode writes --file-perms 0777 --dir-perms 0777 --network-mode --config=C:\Windows\pamdata\rclone.conf
  1. Minimize terminal while working in R

Explore data in R Studio

  1. Open Rstudio (Ignore any update R pop-ups)
  2. Open script through RStudio
  3. Run through script and explore PAMscapes functionality
Triton


  1. Mount data bucket for faster connection to cloud data
    • Open the Windows command prompt (terminal)
    • Copy the code chunk below and enter it into the terminal to mount the bucket that the Triton LTSA and ship detector output are stored in. This code chunk mounts the nefsc-1-detector-output data bucket to the Z: drive.
C:\Windows\rclone-v1.68.1-windows-amd64\rclone mount pamdata-gcs:nefsc-1-detector-output Z: -o UserName=pam_user --vfs-cache-mode writes --file-perms 0777 --dir-perms 0777 --network-mode --config=C:\Windows\pamdata\rclone.conf
  1. If successful, the terminal will return “The service rclone has been started”. Minimize the terminal window and continue with the instructions.
  2. Open triton using the shortcut on the desktop. Be patient for it to open, it can take a few minutes.
  3. Explore the functionality to evaluate detections
    • On Triton Control tab, select “Remoras”, “Ship-Detector”, then “Evaluate detections”

  1. Select the yellow folder with a ‘D’ in the upper left corner then navigate to and open the detector output through the mounted Z: drive. The path to the output is: Z:\MATLAB_VESSEL_FIXED\RAW\NEFSC_GOM\NEFSC_GOM_202105_MDR\

  1. Next, select the yellow folder with an ’L” to open the associated LTSA. Navigate to the same output folder as before and open the LTSA. Z:\MATLAB_VESSEL_FIXED\RAW\NEFSC_GOM\NEFSC_GOM_202105_MDR\
    • Input plot settings on the sh_evaluate tab
      • Start Frequency: 20 Hz
      • End Frequency: 10000 Hz
      • Plot Length: 5 hr (you can adjust this to what you like)
      • Hit Plot if spectrogram doesn’t update
    • Scan data and play with changing classifications.