PAM Windows Workstation

Note

All workstations will be restarted on the 15th of every month at 22:00 UTC for required patching and updates. Please plan your processing accordingly. Each pam-ww can be turned on/off here or in IAP desktop (see Remote Desktop Connections instructions below).

Accessing or updating PAM windows workstation

The pam-ww are provisioned to each individual interested in using them for cloud processing or data validation. To request a pam-ww instance, submit a new issue here and select the “Request a pam-ww” template. If the previous link does not work for you, please submit this form instead. The cloud team will be notified of your request from either form and will comment on the issue or email you when the pam-ww is ready for use.

Required setup before accessing pam–ww for the first time

Configure gcloud

All steps in this section only need to be done once for each computer you want to access the pam-ww from.

  1. To connect to a remote workstation, you have to install software on your primary physical computer. The next steps in this section assume you’re using Windows, alternatives exist for Mac and Linux.

  2. Install Google Cloud SDK on your primary physical computer. In the section titled “Installing the latest gcloud CLI version”, select your personal computer operating system, click “Google Cloud CLI Installer” and follow the steps to configure the client software. See the image below for comparison. Once Cloud SDK is installed, a gcloud terminal window might open. You can close this terminal.

  1. Open the Windows Command Prompt (aka “terminal”) and enter “gcloud auth login”. This will take you to a browser to login with your email password or two-step verification to configure your user credential with gcloud. This step can completed using shells on other operating systems as long as they work with gcloud.

  2. If prompted to set a project, enter the following code chunk:

    gcloud config set project ggn-nmfs-pamdata-prod-1

  3. To confirm authentication was successful, enter ‘gcloud auth list’ in the terminal and check that it returns your NOAA email address.

  4. Once you have confirmed the gcloud auth login was successful, you can close the Windows terminal and move on to the connect steps to access your pam-ww.

Step-by-step access instructions

Turn pam-ww instance on


Prior to connecting, you need to turn your pam-ww instance ‘on’. You can navigate to the link here to see the workstation state and turn it on/off in the browser.

Green checkmarks means on, circle means state is changing (turning on or off), gray means stopped. The three dots on the right are called the ‘hamburger’ menu where you can turn change the on/off state.

Find the instance containing your name, click on the three vertical dots on the right of the row representing that instance, and select ‘Start/Resume’. When you are done working turn your pam-ww off by returning to this page, selecting the hambuger menu, and clicking ‘Stop’. This is also how you turn it off.

Note

Please keep the instance off when not in use, for instance at the end of your workday if you are actively using it. Leaving it on for long periods unnecessarily incurs unneeded cost.

Remote Desktop Connection

Recommended Method: IAP Desktop

We have shifted to recommending IAP Desktop for a smoother experience. This tool provides a singular graphical interface to start, stop, and connect to your workstation.

Note: IAP Desktop is optional open-source software and is not on every approved list. Please use it only as allowed by your local IT software installation policies.

Option 1: Using IAP Desktop (Recommended)

  1. Install and Setup: Download the installer from the IAP Desktop GitHub website and install it on your computer.

  2. Sign In: Open IAP Desktop and click “Sign In with Google”. Use the same email address associated with your PAM project access.

  1. Locate your Workstation: Once logged in, look at the Project Explorer panel on the left. Expand the project tree until you see your workstation name (e.g., firstname-lastname-pam-ww).

  1. Start Your Workstation: You can control the power state of your workstation directly from this list:

  • To Start: Right-click your instance name and select Start. Wait for the “Z” to go away in the instance icon, or refresh.

  • To Stop: When finished working, right-click the instance and select Stop to save resources.

  1. Connect

Right-click on your instance name and select “Connect as User”. IAP Desktop will automatically handle the secure connection.

  • User: Enter pam_user

  • Password: Leave blank (do not supply a password)

  • Click OK.

  1. Subsequent connections

IAP desktop will remember your user name as so “Connect” will usually be suffecient to connect for subsequent connections. Depending on other VM login activity you are engaging in, it/the RDP client may attempt to instead inject your smartcard username (user: first.last). If you experience this it will always be reliable to connect with “Connect as User” so you can opt for pam_user / no password.

Option 2: Command Line Interface (Legacy)

If you are unable to use IAP Desktop, you may continue using the gcloud command line method.

Once your pam-ww instance is running (checked via the console page) or in IAP desktop, follow these steps to connect via Windows Command Prompt.

  1. Open Command Prompt: Open the Windows Command Prompt (cmd.exe).

  2. Run the Connection Command: Copy the code below, replacing firstname-lastname-pam-ww with your specific workstation name.

start gcloud compute start-iap-tunnel firstname-lastname-pam-ww 3389 --local-host-port=localhost:3390 --zone=us-east4-c --project=ggn-nmfs-pamdata-prod-1 && timeout 8 && mstsc /v:localhost:3390

Example for user rebecca.vanhoeck@noaa.gov:

start gcloud compute start-iap-tunnel rebecca-vanhoeck-pam-ww 3389 --local-host-port=localhost:3390 --zone=us-east4-c --project=ggn-nmfs-pamdata-prod-1 && timeout 8 && mstsc /v:localhost:3390
  1. Log In: A new terminal window will open to establish the tunnel, followed by the Remote Desktop (RDP) popup.

  • User: pam_user

  • Password: Leave blank.

  • Click OK.

Note

Troubleshooting:

  • Firewall: If a Windows firewall popup appears, select Cancel.

  • Wrong Account: If prompted for a Google/NOAA account, select “More choices” -> “Use a different account” and enter pam_user.

  • Connection Errors: If the RDP fails to connect, ensure your instance has a green checkmark on the console page or no “Z” on icon in IAP desktop, close the windows, and try Step 2 again.

Alternative Connection Method: If the command above fails, try running this stripped-down command in your terminal: gcloud compute start-iap-tunnel firstname-lastname-pam-ww 3389 --local-host-port=localhost:3390 --zone=us-east4-c --project=ggn-nmfs-pamdata-prod-1

Leave that window open, then manually open the “Remote Desktop Connection” app on your computer and connect to localhost:3390.

Configuring user mounts


NMFS passive acoustic data in the cloud is available through the pam-ww with broad read level access within the desktop shortcut “Mounted_Acoustic_Data”.

To improve connection to the data buckets from some software programs or to conduct data management tasks from the pam-ww, data buckets can be mounted to a letter drive through “user mounts”. In this use case, users are able to apply their full set of write-access permissions to perform administrative tasks on passive acoustic raw data or write to an FMC specific data bucket for analysis. For example, those with data admin permissions will be able to rename, move, or delete audio data from their FMC data bucket.

User mounts are mounted to certain letter drives by default, and each user is free to specify which drive and buckets they would like to mount with their own user permissions. Note that this capability will not grant your user permissions that were not previously granted by cloud admins: ensure you already have the needed permissions for proper functioning.

The steps to configure user mounts are as follows.

  1. Open up the folder called ‘user_drive_mounts_config’ on the desktop.

  1. Open up the files my_mounts.txt and my_mounts_template.txt

  1. Use the syntax specified in my_mounts_template ([drive_letter]:[gcp_bucket_name] …) to specify your desired mounts in my_mounts.txt. For instance, if you were an NEFSC user, you might choose to specify:
N:nefsc-1 M:nefsc-1-detector-outputs

Write your desired drive bucket mounts and then save the my_mounts.txt file.

  1. To get the mounts running, you can click on ‘start_user_mounts.bat’ in the same folder.

  1. The pam-ww will attempt to reuse your authentication as long as it is valid. The first time it runs, and when Google needs to refresh your authentication, the pam-ww will prompt you to login and give user consent with Google. When this happens, enter ‘Y’ in the command window that the ‘start_user_mounts.bat’ opens to complete the browser login and user consent flow.

Ensure you check any boxes provided to you from within the browser process for the authentication to complete successfully.

  1. If successful, you will see a success message in the cmd window. You will also see in Windows Explorer that new drive mounts are now available. Keep this window running as long as you would like the user mounts to be running, and if you accidentally close, you can run the ‘start_user_mounts.bat’ file again to launch the user mounts.

  1. If your needs for user mounts change, you can just go back to the my_mounts.txt file and specify a different set (or no) mounts as desired. Like before, these changes will take effect on login, or when you next run the ‘start_user_mounts.bat’ file.

  1. Unlike the read mounts, the drive mounts substitute the entire name of the bucket for the specified letter. Meaning, if I wanted to access ‘mooring1’ in NEFSC-1/bottom mounted, and the NEFSC-1 bucket was mounted as the N drive, in my application I would path to N:/bottom_mounted/mooring1

Mounts can safely be closed at any time by exiting the running cmd window, logging off, or shutting down the workstation. To re-initate your mounts, run the ‘start_user_mounts.bat’ file.

Streamlining pam-ww access

To more easily open your pam-ww without having to enter code into the command prompt, you can save the access code as a .bat file. Then double click that file to be taken to the Remote Desktop Client to log in.

  1. Copy the below line of code in to a notepad document.
start gcloud compute start-iap-tunnel firstname-lastname-pam-ww 3389 --local-host-port=localhost:3390 --zone=us-east4-c --project=ggn-nmfs-pamdata-prod-1 && timeout 8 && mstsc /v:localhost:3390}
  1. Modify the code to change the “firstname-lastname-pam-ww” to match your name and the name of your workstation.
  2. Save this note as a .bat file by navigating to “File”, then “Save As”. In the “Save as type” dropdown select “All Files”, then name the file something easily identifiable inlcuding the .bat extention in the file name (e.g. “pam_ww_startup.bat”). This will be successful if the file appears in the location you saved it with the file type “Windows Batch File”.

Once the .bat file is created, you can access your pam-ww by turning it on and then double-clicking the .bat file. This will take you to the Remote desktop client for “pam_user” select “OK” and start up should proceed as normal. If opening the file doesn’t work on the first try, simply close the error box and try again. It typically works the second time.

Note

Important: Please remember to turn your pam-ww instance off when it is not actively completing a job. Leaving it on for long periods unnecessarily incurs cost. To turn your instance off, navigate to the link here, click the three vertical dots by your instance and select “Stop”.

Specific software guidance

Anaconda

Anaconda is licensed software that we are not allowed by Terms of Service to redistribute on the pam-ww. Additionally, users need or prefer various flavors of Anaconda, such as GUI anaconda, miniconda, etc. As such it is simpler for users to self install the version of Anaconda they need, and users are able to perform this on their own without admin permissions.

  1. Ensure you are on the latest pam-ww version, and if not, consider an upgrade. This is not mandatory, but is a good idea tp get this out of the way since your Anaconda install will not persist in a newly provisioned pam-ww.
  2. Log in to your pam-ww and open Google Chrome. Navigate to https://www.anaconda.com/download (CLI), or https://www.anaconda.com/products/navigator (GUI), or the appropriate download page for any other Anaconda product as desired. Provide your personal email to access the installer download. Once downloaded, double click and go through the install steps, ensuring you select options to install for a single user and not on the system to avoid the prompt for admin rights.
MATLAB

MATLAB is licensed software that we are not allowed by Terms of Service to redistribute on the pam-ww. It necessitates admin rights, so you will need an admin to assist in it’s install. Steps 1-4 below can be completed independently, but step 5 will require coordination with Dan Woodrich. Please submit a Gitub issue to this repo to schedule a time to meet with Dan.

  1. Ensure you are on the latest pam-ww version by requesting an upgrade if not currently on the latest version. This is mandatory, as we want to minimize the # of MATLAB support sessions as much as possible.
  2. Log in to your pam-ww and open Google Chrome. Navigate to mathworks.com and sign in to access the license center.

  1. Go to your account, select a license, and then go to the “install and activate” tab.

Mathworks allows for you to use a single license on two different computers. If necessary, you can deactivate a computer here to free up a license for use in the cloud.

  1. Select the product you’d like to download. If there are multiple download options, select the first one to “Download and run the installer”, not the the Update. The installer will download to your downloads folder. This is the last step you can complete without having a temporary admin password.

  1. For this step you will need to have an admin on hand, so please create a github issue in this repo or email Dan Woodrich to schedule a session. Admin will stand by during this step to provide you an admin password that will be changed immediately after your use. A video call is required as the admin will need to ensure you are using the temporary credential appropriately.

Once online with the admin, open the installer. Provide the requested fields. The installation process varies slightly for each version of Matlab. A few possibility are shown below. If prompted for the username that will be associated with the license, ensure you specify that pam_user is the Computer login name.

Example from R2023b. If prompted for the computer host id, this can be found by opening up a cmd prompt and running ‘vol’.

  1. After MATLAB has been installed, admin may still be desirable to set search paths that will persist between sessions. The admin can again provide you a temporary admin password for this purpose.

Resolving issues and requesting updates

If you are encountering challenges successfully operating on the pam-ww there are a few troubleshooting steps and ways to request support. In general, a troubleshooting workflow should include (1) reviewing the technical advice listed in the next section to see if there are documented suggestions for your use case; (2) checking which version of the pam-ww you are currently using; (3) Comparing that version with the version history to see if there are updates that might solve your issue; (4) If there is a more recent version, then request an updated pam-ww; and lastly (5) if you are still having issues then submit a bug report or new feature request. Full instructions for each of these steps are below. 

Check pam-ww version and compare to version history

The pam-ww are actively under development as software are tested and bugs are identified and fixed. This means that you may occasionally need to upgrade your pam-ww instance. You can check the current version of your pam-ww by going to the homepage, clicking on the name of your pam-ww, and reviewing ‘Boot disk storage image’ listed in the Basic Information section.

The text under the ‘image’ field indicates the current version of your pam-ww. The pam-ww image name is between the text “pww-disa” and “hardened…”, in this case, “beta-3”. To check whether there are more recent pam-ww versions and identify if the bug you are experiencing has been resolved in one of those versions, can cross reference your current version with pam-ww template change tracking document linked below. Both documents track the changes and are organized in consecutive order where the last entry represents the most recent available version.

Request an upgraded pam-ww

The pam-ww is upgraded through versioning to fix bugs and add new features and software. Currently the default support model is just to replace pam-ww when upgrades are needed. Prior to upgrading your pam-ww:

  1. Ensure all processing jobs on your current pam-ww are complete

  2. Confirm any local files saved on the C: drive of your pam-ww either can be deleted or have been moved to a permanent location. This can either be done by uploading them to google drive, emailing them to yourself, moving them to a detector-output bucket, or moving them to your own folder in the pam-ww-tmp bucket. Note that the pam-ww-tmp bucket is intended to be for temporary storage. Any files there should be regularly deleted or downloaded to your local computer for permanent storage.

  3. Lastly, delete your current pam-ww instance by going to the homepage, clicking the three vertical dots associated with your instance and select ‘Delete’.

To request an upgraded pam-ww submit a new issue here. Click the green “New Issue” button, then select the “Request an upgraded pam-ww” template. If the previous link gives you an error, please use this form to request an upgrade instead. If you had any customized software installed on your instance that you will continue to need on the new pam-ww (eg. Matlab), be sure to include the details of that software in your request. The cloud team will be automatically notified, follow up with any questions, and begin working on your request.

See the next section on requesting new features if there are specific software you’d like installed on your workstation.

Report a bug or request a new feature

Use this form to report a processing issue or request a new feature on the pam-ww. Our cloud development team will get an automated email with your request and will start working on it soon. To check in on the status of your bug/feature request view this smartsheet tracker here.

If there are specific software you’d like installed on your workstation, the cloud team will remote into your instance and install the software on your behalf. In some cases software needs admin privileges as well as personal license credentials with the service (such as MATLAB), in these cases we will set up a shared session to install together.

Technical advice for data processing on the pam-ww

  • Issue: Software is taking a long time or having trouble reading or writing files from the cloud data buckets. Windows based software often have difficulty reading and writing to cloud buckets. By mounting the cloud buckets to a letter drive the software is able to connect to the data in a way that it more easily understood within the Windows OS. Version 10 and on of the pam-ww have implemented user mount configurations to improve reading and writing to the cloud buckets. See the “Configuring user mounts” section above. To manually mount a bucket, open the command terminal within the pam-ww and modify the following code chunk to match the bucket you want to mount and the letter drive you want to mount to. For example, this code chunk mounts the nefsc-1-detector-output bucket to the Z: drive. Leave this terminal window open while your job is processing.
C:\Windows\rclone-v1.68.1-windows-amd64\rclone mount pamdata-gcs:nefsc-1-detector-output Z: -o UserName=pam_user --vfs-cache-mode writes --file-perms 0777 --dir-perms 0777 --network-mode --config=C:\Windows\pamdata\rclone.conf
  • Issue: A multi-day processing job was interrupted. The pam-ww are scheduled to automatically reboot at 22:00 UTC on the 15th of every month for mandotory patching. This will interrupt any processing jobs currently running. Please plan your work accordingly so that jobs will finish before the 15th.

Demo Instructions

Raven

  1. Open Raven Pro 1.6 by selecting the shortcut on the desktop. Either provide a license or use the demo version, then interact with Raven in the same way you would locally. 

  2. Choose data of interest that you want to explore or share

    1. On the desktop, open the “Mounted_Acoustic_Data” file explorer shortcut. 

    2. Explore the data buckets and select data of interest. 

    3. Drag and drop sound files into Raven. Select “Page Sound”, then “Open”

    4. If your group doesn’t have data in the cloud, use one of these recommended deployments: 

      1. Cool humpback song: afsc-1/bottom_mounted/GA19_AU_BT01/12_2019

        • Select files: 191201_040000 - 191201_101000

      2. NPRW gunshot song: afsc-1/bottom_mounted/BS12_AU_PM02-a/08_2012

        • Select files: 120805_032000 - 120805_081000

      3. Very loud humpback: nefsc-1/bottom_mounted/NEFSC_GOM/NEFSC_GOM_202309_USTR02/6556_48kHz_UTC/

        1. Select file: 6556.231112050000

      4. Sperm whales: nefsc-1/bottom_mounted/NEFSC_MA-RI/NEFSC_MA-RI_202205_NS05/NEFSC_MA-RI_202205_NS05_ST/6086_64kHz_UTC/

        1. Select files: 6086.221104023559

  3. Explore acoustic data in Raven: Listen to the sounds, explore changing the spectrogram views, making selections, etc. 

  4. Show and Tell: take turns sharing your screen, talking about the data you chose, and what you’ve found! As you share, others are encouraged to follow along and listen/analyze themselves or explore nearby sections or similar times in other nearby sites. Have fun exploring and talking about interesting acoustic data!

R Studio & PAMscapes


Access required files

  1. Create a personal folder to store temporary files
    • Navigate to the GCP Console, Open the “pam-ww-tmp” bucket, select “Create Folder”, and name it with your First_LastName
  2. In the pam-ww, open google chrome and login to your google drive
    • Open this folder and download the available R script to your personal folder in the pam-ww-tmp bucket that is mounted to the X: Drive
    • The file might automatically save in the downloads folder. If so, open the windows file explorer, copy the file from downloads and paste it in your personal folder in pam-ww-tmp.

Mount Data Buckets for faster access

  1. Mount data bucket of interest
    • Open the Windows command prompt (terminal)
    • Copy the code chunk below and enter it into the terminal to mount the bucket that the soundscape metric data are stored in. This code chunk mounts the nefsc-1 data bucket to the Z: drive.
C:\Windows\rclone-v1.68.1-windows-amd64\rclone mount pamdata-gcs:nefsc-1 Z: -o UserName=pam_user --vfs-cache-mode writes --file-perms 0777 --dir-perms 0777 --network-mode --config=C:\Windows\pamdata\rclone.conf
  1. Minimize terminal while working in R

Explore data in R Studio

  1. Open Rstudio (Ignore any update R pop-ups)
  2. Open script through RStudio
  3. Run through script and explore PAMscapes functionality
Triton


  1. Mount data bucket for faster connection to cloud data
    • Open the Windows command prompt (terminal)
    • Copy the code chunk below and enter it into the terminal to mount the bucket that the Triton LTSA and ship detector output are stored in. This code chunk mounts the nefsc-1-detector-output data bucket to the Z: drive.
C:\Windows\rclone-v1.68.1-windows-amd64\rclone mount pamdata-gcs:nefsc-1-detector-output Z: -o UserName=pam_user --vfs-cache-mode writes --file-perms 0777 --dir-perms 0777 --network-mode --config=C:\Windows\pamdata\rclone.conf
  1. If successful, the terminal will return “The service rclone has been started”. Minimize the terminal window and continue with the instructions.
  2. Open triton using the shortcut on the desktop. Be patient for it to open, it can take a few minutes.
  3. Explore the functionality to evaluate detections
    • On Triton Control tab, select “Remoras”, “Ship-Detector”, then “Evaluate detections”

  1. Select the yellow folder with a ‘D’ in the upper left corner then navigate to and open the detector output through the mounted Z: drive. The path to the output is: Z:\MATLAB_VESSEL_FIXED\RAW\NEFSC_GOM\NEFSC_GOM_202105_MDR\

  1. Next, select the yellow folder with an ’L” to open the associated LTSA. Navigate to the same output folder as before and open the LTSA. Z:\MATLAB_VESSEL_FIXED\RAW\NEFSC_GOM\NEFSC_GOM_202105_MDR\
    • Input plot settings on the sh_evaluate tab
      • Start Frequency: 20 Hz
      • End Frequency: 10000 Hz
      • Plot Length: 5 hr (you can adjust this to what you like)
      • Hit Plot if spectrogram doesn’t update
    • Scan data and play with changing classifications.