Skip to main content

Metadata Submission

Submitting new metadata entries (Analyses) to Song.

CLI Submission Guide

This page documents the basic submission flow, for a more detailed guide see our platform guide on CLI submissions.

Client Installation​

  1. Install the Song-Client:

    To run the song-client using Docker, provide the following environment variables:

    • CLIENT_ACCESS_TOKEN: Valid access token
    • CLIENT_STUDY_ID: Your study ID
    • CLIENT_SERVER_URL: Song server URL

    Use this command to run a Song Client docker container:

    docker run -d -it --name song-client \
        -e CLIENT_ACCESS_TOKEN=${token} \
        -e CLIENT_STUDY_ID=${studyId} \
        -e CLIENT_SERVER_URL=${songServerUrl} \
        --network="host" \
        --mount type=bind,source="$(pwd)",target=/output \
        ghcr.io/overture-stack/song-client:latest

    Replace all ${} placeholders with your environment's values.

    Detailed command breakdown
    • -d -it: Runs container in detached and interactive mode
    • -e CLIENT_ACCESS_TOKEN=${token}: Access token from the platform's auth service
    • -e CLIENT_STUDY_ID=${studyId}: Your specific study ID
    • -e CLIENT_SERVER_URL=${songServerUrl}: Song server URL
    • --network="host": Uses host network stack
    • --mount type=bind,source="$(pwd)",target=/output: Mounts current directory to container's /output

  2. Install the Score-Client:

    To run the score-client using Docker, provide the following environment variables:

    • STORAGE_URL: Score server URL
    • METADATA_URL: Song server URL
    • ACCESSTOKEN: Valid access token

    Use this command to run a Score Client docker container:

    docker run -d --name score-client \
        -e ACCESSTOKEN=${token} \
        -e STORAGE_URL=${scoreServerUrl} \
        -e METADATA_URL=${songServerUrl} \
        --network="host" \
        --platform="linux/amd64" \
        --mount type=bind,source="$(pwd)",target=/output \
        ghcr.io/overture-stack/score:latest

    Replace all ${} placeholders with your environment's values.

    Detailed command breakdown
    • -d: Runs container in detached mode (background)
    • -e ACCESSTOKEN=${token}: Access token from the platform's auth service
    • -e STORAGE_URL=${scoreServerUrl}: Score server URL
    • -e METADATA_URL=${songServerUrl}: Song server URL
    • --network="host": Uses host network stack
    • --platform="linux/amd64": Specifies container platform
    • --mount type=bind,source="$(pwd)",target=/output: Mounts current directory to container's /output

Step 1: Prepare a Payload​

Create a metadata payload conforming to a registered analysis_type schema. For more information, see our section covering Data Model Management.

Step 2: Upload the Metadata Payload​

Use the song-client submit command:

docker exec song-client sh -c "sing submit -f /output/example-payload.json"

Successful submission returns an analysisId:

{
  "analysisId": "a4142a01-1274-45b4-942a-01127465b422",
  "status": "OK"
}

Step 3: Generate a Manifest File​

Use the analysisId to create a manifest for file upload:

docker exec song-client sh -c "sing manifest -a a4142a01-1274-45b4-942a-01127465b422 -f /some/output/dir/manifest.txt -d /submitting/file/directory"
Manifests

A manifest is a file generated by Song that establishes a link between an analysis ID and the data files on your local system, used by Score for file uploads.

Step 4: Upload Data Files​

Use the score-client upload command:

docker exec score-client sh -c "score-client upload --manifest manifest.txt"

For upload issues, try the --force option:

docker exec score-client sh -c "score-client upload --manifest manifest.txt --force"

Step 5: Publish the Analysis​

Set the analysis state to PUBLISHED:

docker exec song-client sh -c "sing publish -a a4142a01-1274-45b4-942a-01127465b422"

For more information, see our documentation on Song publication controls

Search & Exploration

For optimal data querying, use Song with our search & exploration services Maestro, Arranger and Stage. With these components you can index and query and explore Song data through an intuitive search portal.

Troubleshooting​

  • If you encounter connection or internal server errors, have your admin verify that the Song and Score servers are correctly configured.
Support

For technical support, please don't hesitate to reach out through our relevant community support channels.