sign_language_translator.config package

Submodules

• sign_language_translator.config.assets module
  • Assets
    • Assets.ROOT_DIR
    • Assets.FILE_TO_URL
    • Assets.asset_regex_to_urls_file
    • Assets.FILE_TO_URL
    • Assets.ROOT_DIR
    • Assets.asset_regex_to_urls_file
    • Assets.delete()
    • Assets.delete_out_of_date_assets()
    • Assets.download()
    • Assets.extract()
    • Assets.get_ids()
    • Assets.get_path()
    • Assets.get_url()
    • Assets.infer_archive_name()
    • Assets.is_dictionary_video()
    • Assets.load_all_urls()
    • Assets.load_urls()
    • Assets.primary_urls_file
    • Assets.reload()
    • Assets.set_root_dir()
    • Assets.urls_file_dir
• sign_language_translator.config.colors module
  • Colors
    • Colors.BLACK
    • Colors.BLUE
    • Colors.CYAN
    • Colors.DARK_GREY
    • Colors.GREEN
    • Colors.GREY
    • Colors.HOT_PINK
    • Colors.LEAF_GREEN
    • Colors.LIGHT_GREY
    • Colors.MAGENTA
    • Colors.NAVY_BLUE
    • Colors.ORANGE
    • Colors.PINK
    • Colors.PURPLE
    • Colors.RED
    • Colors.YELLOW
    • Colors.gradient()
• sign_language_translator.config.enums module
  • Countries
    • Countries.PAKISTAN
  • ModelCodes
    • ModelCodes.CONCATENATIVE_SYNTHESIS
    • ModelCodes.LOOKUP_UR_FASTTEXT_CC
    • ModelCodes.MEDIAPIPE_POSE_V0_HAND_V1
    • ModelCodes.MEDIAPIPE_POSE_V1_HAND_V1
    • ModelCodes.MEDIAPIPE_POSE_V2_HAND_V1
    • ModelCodes.MIXER_LM_NGRAM_URDU
    • ModelCodes.NGRAM_LM_BIGRAM_NAMES
    • ModelCodes.NGRAM_LM_TRIGRAM_NAMES
    • ModelCodes.NGRAM_LM_UNIGRAM_NAMES
    • ModelCodes.TRANSFORMER_LM_UR_SUPPORTED
  • Organizations
    • Organizations.HFAD
  • SignCollections
    • SignCollections.PK_HFAD_1
    • SignCollections.PK_HFAD_2
  • SignEmbeddingModels
    • SignEmbeddingModels.MEDIAPIPE_WORLD
    • SignEmbeddingModels.MEDIAPIPE_IMAGE
    • SignEmbeddingModels.MEDIAPIPE_IMAGE
    • SignEmbeddingModels.MEDIAPIPE_WORLD
  • SignFormats
    • SignFormats.LANDMARKS
    • SignFormats.VIDEO
  • SignLanguages
    • SignLanguages.PAKISTAN_SIGN_LANGUAGE
  • TextLanguages
    • TextLanguages.ENGLISH
    • TextLanguages.HINDI
    • TextLanguages.URDU
  • normalize_short_code()
• sign_language_translator.config.settings module
  • Settings
    • Settings.AUTO_DOWNLOAD
    • Settings.FILENAME_CONNECTOR
    • Settings.FILENAME_SEPARATOR
    • Settings.SHOW_DOWNLOAD_PROGRESS
• sign_language_translator.config.utils module
  • get_package_version()
  • read_urls()

Module contents

This module provides access to configuration components for the Sign Language Translator package.

Contents: - Assets: Class to manage the assets of the Sign Language Translator package. - enums: Defines enumerations of short codes used in the Sign Language Translator package. - settings: Contains constants and configurations for the Sign Language Translator package. - utils: Contains helper functions for configuration of the package.

class sign_language_translator.config.Assets

Bases: object

Static class for managing assets related to the sign language translator. It wraps around utility functions to automatically handle downloading, extracting, deleting and loading assets.

ROOT_DIR

The root directory path where the sign language datasets & models are stored.

Type:

str

FILE_TO_URL

A dictionary mapping asset filenames to their corresponding URLs.

Type:

Dict[str, str]

asset_regex_to_urls_file

A list of tuples mapping regular expressions matching asset names to URLs files. The UR file must be a JSON containing a key “file_to_url” which maps to an object mapping relative paths of assets to their URLs.

Type:

List[Tuple[str, str]]

set_root_dir(path

str) -> None: Set the SLT resources directory path.

get_ids(filename_or_regex

str) -> List[str]: Get the relative paths of assets matching the given filename_or_regex.

download(filename_or_regex

str, overwrite=False, progress_bar: bool = None, timeout: float = 20.0, leave=True, chunk_size=65536) -> bool: Download assets matching the given filename regex and save them to the appropriate file paths inside the assets root directory.

extract(filename_or_regex

str, archive_name_or_regex: str = None, overwrite=False, download_archive=True) -> List[str]: extract the files matching the argument from an archived dataset into the appropriate location.

delete(filename_or_regex

str) -> None: remove the matching assets from storage and its records from the checksum file.

Example:

import sign_language_translator as slt
# slt.Assets.set_root_dir("~/centralized-slt-assets")

# Archived datasets
ids = slt.Assets.get_ids(r"datasets/.*\.zip")
paths = slt.Assets.download("datasets/pk-hfad-1.landmarks-mediapipe-world-csv.zip")
files = slt.utils.Archive.extract(paths[0], "*.csv")

# Specific file from archive
path = slt.Assets.extract("pk-hfad-1_airplane.landmarks-mediapipe-world.csv", download_archive=True)

# all dictionary videos for numbers
urls = slt.Assets.get_url(r"videos/[a-z-]+_\d+\.mp4")

# download a model
paths = slt.Assets.download(r"models/names-stat-lm-w\d\.json")

# add your own dataset
slt.Assets.FILE_TO_URL.update({"relative/path/to/asset.mp4": "https://...",})

FILE_TO_URL: Dict[str, str] = {'archive-urls.json': 'https://raw.githubusercontent.com/sign-language-translator/sign-language-datasets/d385b3fbdebeba19866c7392e3f5d6f625d5b03e/asset_urls/archive-urls.json', 'datasets/pk-hfad-1.landmarks-mediapipe-image-csv.zip': 'https://github.com/sign-language-translator/sign-language-datasets/releases/download/v0.0.4/pk-hfad-1.landmarks-mediapipe-image-csv.zip', 'datasets/pk-hfad-1.landmarks-mediapipe-world-csv.zip': 'https://github.com/sign-language-translator/sign-language-datasets/releases/download/v0.0.4/pk-hfad-1.landmarks-mediapipe-world-csv.zip', 'datasets/pk-hfad-1.videos-mp4.zip': 'https://github.com/sign-language-translator/sign-language-datasets/releases/download/v0.0.3/pk-hfad-1.videos-mp4.zip', 'datasets/pk-hfad-2.landmarks-mediapipe-image-csv.zip': 'https://github.com/sign-language-translator/sign-language-datasets/releases/download/v0.0.4/pk-hfad-2.landmarks-mediapipe-image-csv.zip', 'datasets/pk-hfad-2.landmarks-mediapipe-world-csv.zip': 'https://github.com/sign-language-translator/sign-language-datasets/releases/download/v0.0.4/pk-hfad-2.landmarks-mediapipe-world-csv.zip', 'datasets/xx-shapes-1.landmarks-testmodel-csv.zip': 'https://github.com/sign-language-translator/sign-language-datasets/releases/download/v0.0.4/xx-shapes-1.landmarks-testmodel-csv.zip', 'datasets/xx-wordless-1.videos-mp4.zip': 'https://github.com/sign-language-translator/sign-language-datasets/releases/download/v0.0.3/xx-wordless-1.videos-mp4.zip', 'extra-urls.json': 'https://raw.githubusercontent.com/sign-language-translator/sign-language-datasets/d385b3fbdebeba19866c7392e3f5d6f625d5b03e/asset_urls/extra-urls.json', 'landmarks/test-landmarks.mediapipe-all.csv': 'https://github.com/sign-language-translator/sign-language-datasets/releases/download/v0.0.4/test-landmarks.mediapipe-all.csv', 'landmarks/test-landmarks.mediapipe-image.csv': 'https://github.com/sign-language-translator/sign-language-datasets/releases/download/v0.0.4/test-landmarks.mediapipe-image.csv', 'landmarks/test-landmarks.mediapipe-world.csv': 'https://github.com/sign-language-translator/sign-language-datasets/releases/download/v0.0.4/test-landmarks.mediapipe-world.csv', 'mapping-schema.json': 'https://raw.githubusercontent.com/sign-language-translator/sign-language-datasets/47c5f28f2395afa6201a39bbdd0c9bc34d9b6612/schemas/mapping-schema.json', 'models/lookup-test-model.pt': 'https://github.com/sign-language-translator/sign-language-datasets/releases/download/v0.0.1/lookup-test-model.pt', 'models/lookup-ur-fasttext-cc.pt': 'https://github.com/sign-language-translator/sign-language-datasets/releases/download/v0.0.1/lookup-ur-fasttext-cc.pt', 'models/mediapipe/hand_landmarker.task': 'https://storage.googleapis.com/mediapipe-models/hand_landmarker/hand_landmarker/float16/latest/hand_landmarker.task', 'models/mediapipe/pose_landmarker_full.task': 'https://storage.googleapis.com/mediapipe-models/pose_landmarker/pose_landmarker_full/float16/latest/pose_landmarker_full.task', 'models/mediapipe/pose_landmarker_heavy.task': 'https://storage.googleapis.com/mediapipe-models/pose_landmarker/pose_landmarker_heavy/float16/latest/pose_landmarker_heavy.task', 'models/mediapipe/pose_landmarker_lite.task': 'https://storage.googleapis.com/mediapipe-models/pose_landmarker/pose_landmarker_lite/float16/latest/pose_landmarker_lite.task', 'models/names-stat-lm-w1.json': 'https://github.com/sign-language-translator/sign-language-datasets/releases/download/v0.0.1/names-stat-lm-w1.json', 'models/names-stat-lm-w2.json': 'https://github.com/sign-language-translator/sign-language-datasets/releases/download/v0.0.1/names-stat-lm-w2.json', 'models/names-stat-lm-w3.json': 'https://github.com/sign-language-translator/sign-language-datasets/releases/download/v0.0.1/names-stat-lm-w3.json', 'models/tlm_14.0M.pt': 'https://github.com/sign-language-translator/sign-language-datasets/releases/download/v0.0.1/tlm_14.0M.pt', 'models/ur-supported-token-unambiguous-mixed-ngram-w1-w6-lm.pkl': 'https://github.com/sign-language-translator/sign-language-datasets/releases/download/v0.0.1/ur-supported-token-unambiguous-mixed-ngram-w1-w6-lm.pkl', 'pk-dictionary-mapping.json': 'https://raw.githubusercontent.com/sign-language-translator/sign-language-datasets/8bb3ac7641c9be94faf75765c8489e5ee758fbba/parallel_texts/pk-dictionary-mapping.json', 'pk-dictionary-urls.json': 'https://raw.githubusercontent.com/sign-language-translator/sign-language-datasets/fcaec746a7bd16c1b27d646409e088e7bce006e2/asset_urls/pk-dictionary-urls.json', 'pk-sentence-mapping.json': 'https://raw.githubusercontent.com/sign-language-translator/sign-language-datasets/8bb3ac7641c9be94faf75765c8489e5ee758fbba/parallel_texts/pk-sentence-mapping.json', 'pk-synthetic-sentence-mapping.json': 'https://raw.githubusercontent.com/sign-language-translator/sign-language-datasets/8bb3ac7641c9be94faf75765c8489e5ee758fbba/parallel_texts/pk-synthetic-sentence-mapping.json', 'text-preprocessing.json': 'https://raw.githubusercontent.com/sign-language-translator/sign-language-datasets/c7fbcb4d53ad4b5c04c43ed62cd98c2a6fa07f63/text-preprocessing.json', 'videos/wordless_wordless.mp4': 'https://github.com/sign-language-translator/sign-language-datasets/releases/download/v0.0.2/wordless_wordless.mp4', 'videos/xx-wordless-1_wordless.mp4': 'https://github.com/sign-language-translator/sign-language-datasets/releases/download/v0.0.2/wordless_wordless.mp4'}

A dictionary mapping asset filenames to their corresponding URLs.

ROOT_DIR: str = '/home/docs/checkouts/readthedocs.org/user_builds/sign-language-translator/checkouts/latest/sign_language_translator/assets'

The root directory path where the sign language datasets & models are stored. Defaults is ‘install_directory/assets’.

asset_regex_to_urls_file: List[Tuple[str, str]] = [('^videos/pk-.*mp4$', 'pk-dictionary-urls.json'), ('.*zip$', 'archive-urls.json'), ('.*', 'extra-urls.json')]

A list of tuples that map regular expressions matching asset names to URLs files containing URLs to that asset group. The regex are tried sequentially so make sure list order is right.

classmethod delete(filename_or_regex: str)

classmethod delete_out_of_date_assets() → None

Delete asset if it is out of date. Currently determined by comparing the download URL of the asset in checksum file with the loaded URL. Does not delete files not present in checksum.

classmethod download(filename_or_regex: str, overwrite=False, timeout: float = 20.0, chunk_size=262144, progress_bar: bool | None = None, leave=True) → List[str]

Downloads package assets matching the given filename regex and saves them to the appropriate file paths.

Parameters:

• filename_or_regex (str) – Relative path or Regular expression to match the desired asset names.

• overwrite (bool, optional) – If False, skips downloading if the resource file already exists. Defaults to False.

• timeout (float, optional) – The maximum number of seconds to wait for a server response. Defaults to 20.0.

• chunk_size (int, optional) – The number of bytes to fetch in each step. Defaults to 256*1024.

• progress_bar (bool, optional) – If True, displays a progress bar during the download. If None, uses the value in slt.Settings.SHOW_DOWNLOAD_PROGRESS. Defaults to None.

• leave (bool, optional) – Wether to leave the progress bar behind after the download. Defaults to True.

Returns:

List of paths to matching files that were downloaded or existed already.

Return type:

List[str]

classmethod extract(filename_or_regex: str, archive_name_or_regex: str | None = None, overwrite=False, progress_bar=True, leave=True, download_archive=True) → List[str]

Extracts assets matching the given filename_or_regex from the archived datasets. The target location is inferred from the archive & asset name and resides inside assets root directory. If the archive name is not provided, it will be inferred from the filename_or_regex, so avoid using vague regex and follow the filename structure. If the archive is not downloaded, it will be downloaded first.

Note

Please use the slt.utils.Archive.extract() function directly if you want deterministic behavior and avoid false guesses.

Parameters:

• filename_or_regex (str) – The filename or regex pattern to match the archive contents.

• archive_name_or_regex (str, optional) – The name or regex pattern of the archive(s) that contains the assets. If None, the function tries guess it from the content name. Defaults to None.

• overwrite (bool, optional) – Flag indicating whether to overwrite existing assets. Defaults to False.

• progress_bar (bool, optional) – Flag indicating whether to display a progress bar during extraction. Defaults to True.

• leave (bool, optional) – Flag indicating whether to leave the progress bar displayed after extraction. Defaults to True.

• download_archive (bool, optional) – Flag indicating whether to download the archive if it is not already downloaded. Defaults to True.

Returns:

A list of paths to the extracted assets.

Return type:

List[str]

classmethod get_ids(filename_or_regex: str) → List[str]

Filters down the loaded Assets.FILE_TO_URLS dictionary and returns the list of asset IDs (relative paths) that match the given filename_or_regex. If no asset_id match the argument, appropriate urls file is loaded from the Assets.asset_regex_to_urls_file list and the function is called recursively.

Parameters:

filename_or_regex (str) – The filename or regex to match against asset IDs.

Returns:

List of matching asset IDs.

Return type:

List[str]

classmethod get_path(filename_or_regex: str) → List[str]

Filters down the loaded Assets.FILE_TO_URLS dictionary and returns the list of file paths corresponding to the asset_ids matching given filename_or_regex.

Parameters:

filename_or_regex (str) – The filename or regex to match against asset IDs.

Returns:

List of matching file paths.

Return type:

List[str]

classmethod get_url(filename_or_regex: str) → List[str]

Filters down the loaded Assets.FILE_TO_URLS dictionary and returns the list of URLs corresponding to the asset_ids matching given filename_or_regex.

Parameters:

filename_or_regex (str) – The filename or regex to match against asset IDs.

Returns:

List of matching URLs.

Return type:

List[str]

classmethod infer_archive_name(filename_or_regex: str) → str

Infers the archive name/regex that should contain the given asset based on the provided filename_or_regex argument. Please follow the naming convention to avoid false guesses i.e. use all allowed special symbols [“/”, “-”, “_”, “.”, r”.”] in the right places.

Parameters:

filename_or_regex (str) – The asset filename or regex from which its containing archive name must be inferred.

Returns:

A regex pattern that matches the archive name which should contain the given asset.

Return type:

str

classmethod is_dictionary_video(filename: str) → bool

Class method to check if the given filename is a dictionary video. Checks the folder name, extension & direct URL.

Parameters:

filename (str) – The asset ID to check. (e.g. ‘videos/pk-hfad-1_airplane.mp4’)

Returns:

True if the filename represents a dictionary video, False otherwise.

Return type:

bool

classmethod load_all_urls() → None

Load all URL files into the Assets.FILE_TO_URL dictionary.

classmethod load_urls(filename: str) → None

Load URLs from the specified file into the Assets.FILE_TO_URL dictionary.

Parameters:

filename (str) – The name of the URLs file to load.

primary_urls_file: str = 'urls.json'

The name of the first URLs file that is loaded by default and must contain links to other url files. Defaults is ‘urls.json’. Note: all url filenames must end with ‘urls.json’.

classmethod reload() → None

Clear the cache and read the URL files again.

classmethod set_root_dir(path: str) → None

Set the SLT resources directory path. Helpful when using custom datasets, or when data must be stored outside install directory at a centralized location. (e.g hosted on cloud and mounted on disk). By default, resources are downloaded to the ‘install_directory/assets’.

Parameters:

path (str) – The path to the assets, datasets or models directory.

Raises:

ValueError – If the provided path is not a directory.

urls_file_dir: str = '/home/docs/checkouts/readthedocs.org/user_builds/sign-language-translator/checkouts/latest/sign_language_translator/config'

The directory path where the URLs files are stored. Defaults is ‘install_directory/sign_language_translator/config’.

class sign_language_translator.config.Colors(BLACK: Tuple[int, int, int] = (0, 0, 0), NAVY_BLUE: Tuple[int, int, int] = (0, 0, 255), GREEN: Tuple[int, int, int] = (0, 255, 0), RED: Tuple[int, int, int] = (255, 0, 0), CYAN: Tuple[int, int, int] = (0, 255, 255), MAGENTA: Tuple[int, int, int] = (255, 0, 255), YELLOW: Tuple[int, int, int] = (255, 255, 0), ORANGE: Tuple[int, int, int] = (255, 127, 0), PURPLE: Tuple[int, int, int] = (127, 0, 255), PINK: Tuple[int, int, int] = (255, 127, 255), HOT_PINK: Tuple[int, int, int] = (255, 0, 127), BLUE: Tuple[int, int, int] = (0, 127, 255), LEAF_GREEN: Tuple[int, int, int] = (127, 255, 0), DARK_GREY: Tuple[int, int, int] = (63, 63, 63), GREY: Tuple[int, int, int] = (127, 127, 127), LIGHT_GREY: Tuple[int, int, int] = (191, 191, 191))

Bases: object

A class to represent a set of predefined colors and provide utility functions related to colors.

BLACK: Tuple[int, int, int] = (0, 0, 0)

BLUE: Tuple[int, int, int] = (0, 127, 255)

CYAN: Tuple[int, int, int] = (0, 255, 255)

DARK_GREY: Tuple[int, int, int] = (63, 63, 63)

GREEN: Tuple[int, int, int] = (0, 255, 0)

GREY: Tuple[int, int, int] = (127, 127, 127)

HOT_PINK: Tuple[int, int, int] = (255, 0, 127)

LEAF_GREEN: Tuple[int, int, int] = (127, 255, 0)

LIGHT_GREY: Tuple[int, int, int] = (191, 191, 191)

MAGENTA: Tuple[int, int, int] = (255, 0, 255)

NAVY_BLUE: Tuple[int, int, int] = (0, 0, 255)

ORANGE: Tuple[int, int, int] = (255, 127, 0)

PINK: Tuple[int, int, int] = (255, 127, 255)

PURPLE: Tuple[int, int, int] = (127, 0, 255)

RED: Tuple[int, int, int] = (255, 0, 0)

YELLOW: Tuple[int, int, int] = (255, 255, 0)

static gradient(n: int, colors: ~typing.Sequence[~typing.Tuple[float | int, float | int, float | int]] = ((255, 0, 127), (0, 127, 255), (255, 0, 127)), endpoint=False, dtype=<class 'int'>) → List[Tuple[float, float, float]]

Generate a gradient of colors by linearly interpolating between a sequence of colors.

Parameters:

• n (int) – The number of colors to generate in the gradient.

• colors (Sequence[Tuple[Union[float, int], Union[float, int], Union[float, int]]], optional) – A sequence of RGB tuples to interpolate between. Defaults to (HOT_PINK, BLUE, HOT_PINK).

• endpoint (bool, optional) – If True, the last value in the colors sequence is included. Defaults to False.

• dtype (Union[type, str], optional) – The desired data type of the output colors. Defaults to int.

Returns:

A list of RGB tuples representing the gradient.

Return type:

List[Tuple[float, float, float]]

Example

>>> Colors.gradient(5, colors = [(128, 0, 0), (0, 0, 128)], endpoint=True)
[[128, 0, 0], [96, 0, 32], [64, 0, 64], [32, 0, 96], [0, 0, 128]]

class sign_language_translator.config.Settings

Bases: object

Class containing settings and configuration parameters for the sign language translator library.

AUTO_DOWNLOAD = True

A flag indicating whether automatic downloading of missing dataset files is enabled.

FILENAME_CONNECTOR = '-'

The connector used in filenames to join parts of same attribute.

FILENAME_SEPARATOR = '_'

The separator used in dataset filenames to separate different attributes.

SHOW_DOWNLOAD_PROGRESS = True

A flag indicating whether the download progress should be shown by default.