Documentation for `aalibrary`

Modules:

Name	Description
`config`	Used for storing environment-specific settings such as database URIs and
`ices_ship_names`	This file contains the code to parse through the ICES API found here:
`ingestion`	This file contains functions used to ingest Active Acoustics data into GCP
`conversion`	This file is used to store conversion functions for the AALibrary.
`metadata`	This file contains functions that have to do with metadata.
`queries`	This script contains classes that have SQL queries used for interaction

`config`

Used for storing environment-specific settings such as database URIs and such.

`ices_ship_names`

This file contains the code to parse through the ICES API found here: https://vocab.ices.dk/?ref=315 Specifically the SHIPC platform code which refers to ship names.

Functions:

Name	Description
`get_all_ices_ship_codes_and_names`	Gets all of the ices ship codes and their corresponding names in a
`get_all_ices_ship_names`	Gets all of the ICES ship names. You can normalize them to our standards
`get_all_ship_info`	Gets all of the ship's info from the following URL:
`get_ices_code_from_ship_name`	Gets the ICES Code for a ship given a ship's name.

`get_all_ices_ship_codes_and_names(normalize_ship_names=False)`

Gets all of the ices ship codes and their corresponding names in a dictionary format. The keys are the ICES code, and the name is the value.

Parameters:

Name	Type	Description	Default
`normalize_ship_names`	`bool`	Whether or not to format the ship name according to our own standards. Defaults to False.	`False`

Returns:

Name	Type	Description
`dict`	`dict`	A dict with all of the ICES ships. The keys are the ICES code, and the name is the value.

Source code in src\aalibrary\ices_ship_names.py

def get_all_ices_ship_codes_and_names(
    normalize_ship_names: bool = False,
) -> dict:
    """Gets all of the ices ship codes and their corresponding names in a
    dictionary format. The keys are the ICES code, and the name is the value.

    Args:
        normalize_ship_names (bool, optional): Whether or not to format the
            ship name according to our own standards. Defaults to False.

    Returns:
        dict: A dict with all of the ICES ships. The keys are the ICES code,
            and the name is the value.
    """

    all_ship_info = get_all_ship_info()
    all_ship_codes_and_names = {}
    for ship_info in all_ship_info:
        all_ship_codes_and_names[ship_info["key"]] = ship_info["description"]

    if normalize_ship_names:
        all_ship_codes_and_names = {
            code: normalize_ship_name(name)
            for code, name in all_ship_codes_and_names.items()
        }

    return all_ship_codes_and_names

`get_all_ices_ship_names(normalize_ship_names=False)`

Gets all of the ICES ship names. You can normalize them to our standards if you wish.

Parameters:

Name	Type	Description	Default
`normalize_ship_names`	`bool`	Whether or not to format the ship name according to our own standards. Defaults to False.	`False`

Returns:

Name	Type	Description
`List`	`List`	A list containing strings of all of the ship names.

Source code in src\aalibrary\ices_ship_names.py

def get_all_ices_ship_names(normalize_ship_names: bool = False) -> List:
    """Gets all of the ICES ship names. You can normalize them to our standards
    if you wish.

    Args:
        normalize_ship_names (bool, optional): Whether or not to format the
            ship name according to our own standards. Defaults to False.

    Returns:
        List: A list containing strings of all of the ship names.
    """

    all_ship_info = get_all_ship_info()
    all_ship_names = []
    for ship_info in all_ship_info:
        # Here `ship_info` is a dict
        all_ship_names.append(ship_info["description"])
    if normalize_ship_names:
        all_ship_names = [
            normalize_ship_name(ship_name=ship_name)
            for ship_name in all_ship_names
        ]

    return all_ship_names

`get_all_ship_info()`

Gets all of the ship's info from the following URL: https:/vocab.ices.dk/services/api/Code/7f9a91e1-fb57-464a-8eb0-697e4b0235b5

Returns:

Name	Type	Description
`List`	`List`	A list with dicts of all the ships, including name, ices code, uuids and other fields.

Source code in src\aalibrary\ices_ship_names.py

def get_all_ship_info() -> List:
    """Gets all of the ship's info from the following URL:
    https:/vocab.ices.dk/services/api/Code/7f9a91e1-fb57-464a-8eb0-697e4b0235b5


    Returns:
        List: A list with dicts of all the ships, including name, ices code,
            uuids and other fields.
    """

    response = requests.get(
        url=(
            "https://vocab.ices.dk/services/api/Code/"
            "7f9a91e1-fb57-464a-8eb0-697e4b0235b5"
        ),
        timeout=10
    )
    all_ship_info = response.json()

    return all_ship_info

`get_ices_code_from_ship_name(ship_name='', is_normalized=False)`

Gets the ICES Code for a ship given a ship's name.

Parameters:

Name	Type	Description	Default
`ship_name`	`str`	The ship name string. Defaults to "".	`''`
`is_normalized`	`bool`	Whether or not the ship name is already normalized according to aalibrary standards. Defaults to False.	`False`

Returns:

Name	Type	Description
`str`	`str`	The ICES Code if one has been found. Empty string if it has not.

Source code in src\aalibrary\ices_ship_names.py

def get_ices_code_from_ship_name(
    ship_name: str = "", is_normalized: bool = False
) -> str:
    """Gets the ICES Code for a ship given a ship's name.

    Args:
        ship_name (str, optional): The ship name string. Defaults to "".
        is_normalized (bool, optional): Whether or not the ship name is already
            normalized according to aalibrary standards. Defaults to False.

    Returns:
        str: The ICES Code if one has been found. Empty string if it has not.
    """

    # Get all of the ship codes and names.
    all_codes_and_names = get_all_ices_ship_codes_and_names(
        normalize_ship_names=is_normalized
    )
    # Reverse it to make the ship names the keys.
    all_codes_and_names = {v: k for k, v in all_codes_and_names.items()}
    valid_ices_ship_names = list(all_codes_and_names.keys())
    # Try to find the correct ICES code based on the ship name.
    try:
        return all_codes_and_names[ship_name]
    except KeyError:
        # Here the ship name does not exactly match any in the ICES DB.
        # Check for spell check using custom list
        spell_check_list = get_close_matches(
            ship_name, valid_ices_ship_names, n=3, cutoff=0.6
        )
        if len(spell_check_list) > 0:
            print(
                f"This `ship_name` {ship_name} does not"
                " exist in the ICES database. Did you mean one of the"
                f" following?\n{spell_check_list}"
            )
        else:
            print(
                f"This `ship_name` {ship_name} does not"
                " exist in the ICES database. A close match could not be "
                "found."
            )
        return ""

`ingestion`

This file contains functions used to ingest Active Acoustics data into GCP from various sources such as AWS buckets and Azure Data Lake.

Functions:

Name	Description
`download_file_from_azure_directory`	Downloads a single file from an azure directory using the
`download_netcdf_file`	ENTRYPOINT FOR END-USERS
`download_raw_file`	ENTRYPOINT FOR END-USERS
`download_raw_file_from_azure`	ENTRYPOINT FOR END-USERS
`download_raw_file_from_ncei`	ENTRYPOINT FOR END-USERS
`download_specific_file_from_azure`	Creates a DataLakeFileClient and downloads a specific file from
`download_survey_from_ncei`	Downloads an entire survey from NCEI to a local directory while
`find_and_upload_survey_metadata_from_s3`	Finds the metadata that is associated with a particular survey in s3,
`find_data_source_for_file`	Finds the data source of a given filename by checking all possible data

`download_file_from_azure_directory(directory_client, file_system='testcontainer', download_directory='./', file_path='')`

Downloads a single file from an azure directory using the DataLakeDirectoryClient. Useful for numerous operations, as authentication is only required once for the creation of each DataLakeDirectoryClient.

Parameters:

Name	Type	Description	Default
`directory_client`	`DataLakeDirectoryClient`	The DataLakeDirectoryClient that will be used to connect to a download from an azure file system in the data lake.	required
`file_system`	`str`	The file system (container) you wish to download your file from. Defaults to "testcontainer" for testing purposes.	`'testcontainer'`
`download_directory`	`str`	The local directory you want to download to. Defaults to "./".	`'./'`
`file_path`	`str`	The file path you want to download.	`''`