Tutorial: Building a Configuration-Driven CLI App¶

Abstract

We will build a simple, type-safe, automatically generated command-line application that can be configured via layered YAML files and command-line arguments and has strong type-safety.

Step 1: Project Setup¶

Create a project directory with the following structure:

dracon_tutorial/
├── config/
│   ├── base.yaml
│   ├── prod.yaml
│   └── db_user.secret
├── models.py
└── main.py

Step 2: Define Configuration Models (`models.py`)¶

We use Pydantic models to define the structure and types of our configuration. Dracon uses these models for validation and CLI argument generation.

models.py

from pydantic import BaseModel, Field
from typing import Annotated, Literal, Optional
from dracon import Arg, DeferredNode, construct


class DatabaseConfig(BaseModel):
    """Configuration for the database connection."""

    host: str = 'localhost'
    port: int = 5432
    username: str  # Made required for the example
    password: str  # Made required for the example


class AppConfig(BaseModel):
    """Main application configuration model."""

    input_path: Annotated[
        str,
        Arg(help="Example of positional argument.", positional=True),
    ] = './'

    environment: Annotated[
        Literal['dev', 'prod', 'test'],
        Arg(short='e', help="Deployment environment."),
    ]
    log_level: Annotated[
        Literal["DEBUG", "INFO", "WARNING", "ERROR"], Arg(help="Logging level")
    ] = "INFO"
    workers: Annotated[int, Arg(help="Number of worker processes.")] = 1
    # Nested model, can be populated from YAML/CLI. Uses default_factory for Pydantic v2 best practice.
    database: Annotated[DatabaseConfig, Arg(help="Database configuration.")] = Field(
        default_factory=DatabaseConfig
    )
    # Output path depends on runtime context, marked as DeferredNode.
    output_path: Annotated[DeferredNode[str], Arg(help="Path for output files.")] = (
        "/tmp/dracon_output"  # Provide a default
    )

    def process_data(self):
        """Example method demonstrating use of the loaded configuration."""
        print("-" * 20)
        print(f"Processing for environment: {self.environment}")
        print(f"Using Database:")
        print(f"  Host: {self.database.host}")
        print(f"  Port: {self.database.port}")
        print(f"  User: {self.database.username}")
        # print(f"  Password: {'*' * len(self.database.password)}") # Avoid printing password
        print(f"Settings:")
        print(f"  Workers: {self.workers}")
        print(f"  Log Level: {self.log_level}")

        # The output_path is a DeferredNode. We need to call construct()
        # to get the final value, providing any necessary context.
        print("Constructing output path...")
        final_output = construct(
            self.output_path, context={'computed_runtime_value': self.generate_unique_id()}
        )
        print(f"  Output Path: {final_output}")
        print("-" * 20)

        # ... actual application logic would go here ...

    def generate_unique_id(self) -> str:
        """Example helper to generate a value based on current config state."""
        from time import time

        # In a real app, this might involve more complex logic or external calls
        return f"{self.environment}-{self.database.host}-{self.workers}-{int(time())}"

DatabaseConfig: Defines database connection details with defaults.
AppConfig: Defines the main application settings.
Annotated and dracon.Arg are used to customize CLI arguments (short flags -e, help text, marking environment as required).
database uses Field(default_factory=...) for the nested Pydantic model.
output_path is marked as DeferredNode[str]. This tells Dracon its final value depends on runtime context and construction should be delayed until dracon.construct() is called.
process_data: An example method showing how to use the configuration, including constructing the deferred output_path.
generate_unique_id: A helper simulating runtime value generation.

Step 3: Create Base Configuration (`config/base.yaml`)¶

This file defines default settings and can load sensitive data from other sources.

config/base.yaml

# Optional: You can tag the root with the Pydantic model,
# although it's often done in the loading script or CLI definition.
# !AppConfig

log_level: ${getenv('LOG_LEVEL', 'INFO')} # Use env var LOG_LEVEL or default to INFO

database:
  # Dynamically set host based on the 'environment' key IN THE FINAL config object
  # This uses lazy evaluation (${...}) and a reference (@/) to another key.
  host: "db.${@/environment}.local"
  port: 5432 # Default port
  # Include sensitive data from another file relative to this one.
  # $DIR is automatically provided by Dracon's file loader.
  username: !include file:$DIR/db_user.secret
  # Load password directly from an environment variable during composition.
  password: !include env:DB_PASS

# Default output path, potentially overridden by other configs or CLI.
# Uses interpolation needing runtime context provided via construct().
output_path: "/data/outputs/${computed_runtime_value}"

# Default workers, can be overridden
workers: 1

log_level: Uses lazy interpolation (${...}) to read the LOG_LEVEL environment variable, falling back to "INFO".
database.host: Uses a cross-reference (@/environment) to dynamically build the host based on the final environment value after all merging/overrides.
database.username: Uses !include file:$DIR/... to load the username from db_user.secret located in the same directory ($DIR).
database.password: Uses !include env:DB_PASS to load the password directly from the DB_PASS environment variable during the composition phase.
output_path: Defines the structure, but ${computed_runtime_value} needs to be provided later via construct().

Step 4: Create Production Overrides (`config/prod.yaml`)¶

This file overrides specific settings for the production environment and merges the base configuration.

config/prod.yaml

# Define overrides specific to the production environment
environment: prod # Explicitly set environment for this config layer
log_level: WARNING # Override log level for prod
workers: 4 # Increase workers for prod

database:
  # Only override specific DB fields needed for production
  host: "db.prod.svc.cluster.local" # Production database host
  username: prod_db_user # Production database user
  # 'port' and 'password' will be inherited from base.yaml merge below

# Override output path format for production
output_path: "/data/prod/${computed_runtime_value}/output"

# Include and merge base.yaml *after* defining overrides.
# Merge strategy <<{<+}:
# {<+}: Dictionary merge: recursive ({+}), new values (from base.yaml here) win (<).
# This means if a key exists in both prod.yaml and base.yaml, the one from base.yaml
# will be kept during the merge, *unless* it's a nested dictionary, in which case
# the dictionaries are merged recursively following the same rule.
# Lists are replaced by default (new wins). See Merging docs for details.

<<{>+}: !include file:$DIR/base.yaml # Note: EXISTING values (from prod.yaml) win over new values (from base.yaml)

It explicitly sets environment, log_level, workers, database.host, and database.username.
The <<{<+}: !include file:base.yaml line is crucial:
<<: indicates a merge operation.
!include file:base.yaml specifies the source to merge (our base config).
{<+} defines the merge strategy: recursively merge dictionaries (+), letting values from the new source (base.yaml in this case) win conflicts (<). This means defaults from base.yaml will be used if not defined in prod.yaml. If we wanted prod.yaml values to always take precedence, we would use {>+}.

Step 5: Create Secret File (`config/db_user.secret`)¶

A simple text file holding the base database username.

config/db_user.secret

base_user

Step 6: Create the CLI Script (`main.py`)¶

This script uses Dracon to create the CLI program based on AppConfig.

main.py

import sys
import os
from dracon import make_program

# Ensure models.py is in the same directory or Python path
from models import AppConfig, DatabaseConfig

# Create the CLI program from the Pydantic model
# Dracon automatically generates args for 'environment', 'log_level', 'workers', etc.
# It also handles nested fields like 'database.host', creating --database.host
program = make_program(
    AppConfig,  # The Pydantic model defining the configuration and CLI args
    name="my-cool-app",
    description="My cool application using Dracon for config and CLI.",
    # Provide models to Dracon's context so it can construct them
    # when encountering tags like !AppConfig or !DatabaseConfig in YAML files.
    context={
        'AppConfig': AppConfig,
        'DatabaseConfig': DatabaseConfig,
        # You can add other functions or variables here to make them
        # available inside ${...} expressions in your YAML files.
        # 'my_helper_func': some_function,
    },
)

if __name__ == "__main__":
    # Ensure the 'config' directory is accessible relative to this script
    script_dir = os.path.dirname(os.path.abspath(__file__))
    os.chdir(script_dir)  # Change CWD to script dir for relative paths

    print()
    print()
    print()
    # program.parse_args handles:
    # 1. Parsing known arguments based on AppConfig and Arg annotations.
    # 2. Identifying config files specified with '+' (e.g., +config/prod.yaml).
    # 3. Loading and merging these config files sequentially.
    # 4. Applying direct CLI overrides (e.g., --workers 8).
    # 5. Handling --define.VAR=value for context variables.
    # 6. Validating the final merged configuration against AppConfig.
    # 7. Returning the validated AppConfig instance and a dict of raw args.
    cli_config, raw_args = program.parse_args(sys.argv[1:])

    print("\n--- Successfully Parsed Config ---")
    # cli_config is now a fully populated and validated AppConfig instance
    # ready to be used by the application.
    cli_config.process_data()

    print("\n--- Raw Arguments Provided ---")
    print(raw_args)  # Shows the arguments as parsed by the CLI layer

make_program(AppConfig, ...): This is the core of CLI generation. Dracon inspects AppConfig and the Arg annotations.
context={...}: We pass the Pydantic models to the loader's context so Dracon knows how to construct !AppConfig or !DatabaseConfig tags if encountered in YAML (though not strictly needed in this specific example's YAML, it's good practice).
program.parse_args(sys.argv[1:]): This function does the heavy lifting:
Parses standard CLI arguments (-e, --workers, etc.).
Detects configuration files prefixed with + (e.g., +config/prod.yaml).
Loads and merges the base Pydantic defaults, the specified config files (in order), and CLI overrides according to Dracon's precedence rules.
Handles --define.VAR=value arguments.
Validates the final configuration object against AppConfig.
Returns the validated AppConfig instance (cli_config).
cli_config.process_data(): We call our application logic using the fully configured object.

Step 7: Run the Application¶

Now, let's run it with different configurations:

Show Help: See the automatically generated help message.
```
python main.py --help
```
(You should see output similar to the screenshot in the Introduction)

Run with Development Environment: Requires the DB_PASS environment variable.

export DB_PASS="dev_secret_shhh"
python main.py +config/base.yaml -e dev

Expected Output Snippets:

Processing for environment: dev
Using Database:
  Host: db.dev.local
  User: base_user
Settings:
  Workers: 1
  Log Level: INFO
Constructing output path...
  Output Path: /data/outputs/dev-db.dev.local-1-xxxxxxxxx

Note

The host is db.dev.local because environment became 'dev'. The timestamp in the output path will vary.)

Run with Production Config & Overrides: Load prod.yaml and override workers. Also set LOG_LEVEL via environment.

export LOG_LEVEL=DEBUG
export DB_PASS="prod_secret_much_safer"
python main.py +config/prod.yaml --workers 8

Expected Output Snippets:

Processing for environment: production
Using Database:
  Host: db.prod.svc.cluster.local
  User: prod_db_user
Settings:
  Workers: 8
  Log Level: DEBUG
Constructing output path...
  Output Path: /data/prod/production-db.prod.svc.cluster.local-8-xxxxxxxxx

Note

environment, host, user, and initial workers came from prod.yaml. log_level came from the environment variable. workers=8 came from the CLI override. The output path format is also from prod.yaml.)

Override Nested Value from File:

echo "cli_override_user" > /tmp/override_user.secret
python main.py +config/prod.yaml --database.username +/tmp/override_user.secret

(Expected: The database username will be cli_override_user)

Tutorial: Building a Configuration-Driven CLI App¶

Step 1: Project Setup¶

Step 2: Define Configuration Models (models.py)¶

Step 3: Create Base Configuration (config/base.yaml)¶

Step 4: Create Production Overrides (config/prod.yaml)¶

Step 5: Create Secret File (config/db_user.secret)¶

Step 6: Create the CLI Script (main.py)¶