Home
User Guide
Developer Guide
Home
User Guide
Developer Guide

User Guide

What is panconf?

panconf is a CLI tool that generates configuration files from multiple sources. It supports:

  • Multiple input formats: JSON, JSON5, YAML, TOML
  • Multiple input sources: local files, URLs, glob patterns, commands, inline data
  • Multiple output formats: JSON, YAML, TOML
  • Multiple transforms: deep merge, tree merge, Go templates, jq, external commands
  • Multi-document nodes: each node can contain multiple named documents
  • DAG-based processing: nodes are processed in dependency order
  • Intermediate nodes: transform nodes without output files that can be referenced by other nodes
  • Template expansion: reference nodes, environment variables, tilde paths
  • Watch mode: automatically regenerate when input files change

Installation

go install github.com/gforien/panconf/cmd/panconf@latest

Quick Start

  1. Create a configuration file panconf.yaml:
node:
  base:
    from_file: base.json
  overrides:
    from_file: overrides.json
  config:
    to_file: config.json
    transform:
      deepMerge:
        - "{{.base}}"
        - "{{.overrides}}"
  1. Create your input files:
// base.json
{
  "name": "myapp",
  "settings": {
    "debug": false,
    "port": 8080
  }
}
// overrides.json
{
  "settings": {
    "debug": true,
    "timeout": 30
  }
}
  1. Run panconf:
panconf run

Or watch for changes:

panconf watch

Output Format Options

When writing to stdout (to_file: "-"), output is JSON by default with pretty-printing and colorization.

panconf run -ojson    # JSON (default)
panconf run -oyaml    # YAML
panconf run -otoml    # TOML
panconf run --no-color  # Disable colorization
  1. Result in config.json:
{
  "name": "myapp",
  "settings": {
    "debug": true,
    "port": 8080,
    "timeout": 30
  }
}

Configuration File

panconf looks for configuration in the current directory, checking these files in order:

  1. panconf.jsonnet (Jsonnet - programmable configuration with functions, imports, etc.)
  2. panconf.json5 (JSON5 - JSON with comments, trailing commas, unquoted keys, etc.)
  3. panconf.json
  4. panconf.toml
  5. panconf.yaml

Use panconf init to create a default panconf.json5 with helpful comments.

Schema

node:
  <name>:                      # Named node
    # INPUT NODE (exactly one from_* field):
    from_file: <path>          # Single file (doc named "doc")
    from_file: [<path>, ...]   # Multiple files (docs named by stem)
    from_file:                 # Named files
      - <name>: <path>
    from_url: <url>            # Remote URL
    from_glob: <pattern>       # Glob pattern (docs named by stem)
    from_cmd: <command>        # Command output (string or array)
    from_expr:                 # Inline data (embedded YAML/JSON)
      key: value

    # TRANSFORM NODE (has transform, optional to_file):
    to_file: <path>            # Output path ("-" for stdout, optional)
    transform:
      # Exactly ONE of these:
      deepMerge: [<refs>]      # Deep merge (permissive)
      treeMerge: [<refs>]      # Tree merge (strict)
      gotmpl: [<template>, <data>...]  # Go template from file
      gotmpl_inline:           # Inline Go template
        tmpl: <template>
      jq: <expression>         # jq expression
      jsonnet: <expression>    # Jsonnet expression
      cmd: [<args>]            # External command

Node Types

There are two types of nodes:

Input Nodes

Input nodes load data from external sources. They have exactly one from_* field:

node:
  config:
    from_file: config.json

Transform Nodes

Transform nodes process data using a transform. They have a transform: field and optionally a to_file: field:

node:
  merged:
    # No to_file = intermediate node (not written to disk)
    transform:
      deepMerge: ["{{.base}}", "{{.env}}"]

  output:
    to_file: result.json
    transform:
      deepMerge: ["{{.merged}}"]

Multi-Document Nodes

Each node in panconf contains a map of named documents. This allows a single node to hold multiple pieces of data.

Single-Document Shorthand

When a node has only one document (single file, single URL, etc.), it's automatically named "doc" and can be accessed with shorthand:

node:
  config:
    from_file: config.json    # Single document named "doc"

Access as {{.config}} (shorthand) or {{.config.doc}} (explicit).

Multiple Files

Load multiple files into a single node:

node:
  inputs:
    from_file:
      - base.json           # Document named "base"
      - override.json       # Document named "override"

Access individual documents: {{.inputs.base}}, {{.inputs.override}}

In deepMerge, {{.inputs}} expands to all documents in sorted order.

Explicit Naming

Override automatic naming with explicit names:

node:
  settings:
    from_file:
      - defaults: configs/defaults.json    # Named "defaults"
      - custom: configs/overrides.json     # Named "custom"

Access as {{.settings.defaults}} and {{.settings.custom}}.

Glob Documents

Glob patterns create documents named by the filename stem:

node:
  configs:
    from_glob: "configs/*.json"

If files are configs/database.json and configs/cache.json, access as {{.configs.database}} and {{.configs.cache}}.

Input Sources

File Input

Load configuration from local files:

node:
  # Single file
  config:
    from_file: config.json

  # Multiple files
  layers:
    from_file:
      - base.yaml
      - env.yaml
      - local.yaml

  # With explicit names
  settings:
    from_file:
      - defaults: path/to/defaults.json
      - custom: path/to/custom.json

Supported formats: .json, .jsonnet, .libsonnet, .yaml, .yml, .toml, .gotmpl, .tmpl

Jsonnet Input

Load configuration from Jsonnet files with full support for imports and the standard library:

node:
  # Simple jsonnet file
  config:
    from_file: config.jsonnet

  # Jsonnet with imports (imports resolved relative to file)
  app:
    from_file: app/config.jsonnet

config.jsonnet:

local defaults = import 'lib/defaults.libsonnet';

{
  name: "myapp",
  port: defaults.port,
  replicas: std.parseInt(std.extVar("REPLICAS")),
  items: std.range(1, 5),
}

Jsonnet files are evaluated using go-jsonnet. The resulting JSON is then available like any other input.

URL Input

Fetch configuration from remote URLs:

node:
  # Single URL (document named "doc")
  remote:
    from_url: https://example.com/config.json

  # Multiple URLs (documents named by URL filename stem)
  remotes:
    from_url:
      - https://example.com/base.json       # doc named "base"
      - https://example.com/override.json   # doc named "override"

  # With explicit names
  configs:
    from_url:
      - defaults: https://example.com/defaults.json
      - custom: https://internal.api/config.json

The file is downloaded to a temporary location for processing. Format is detected from URL extension or Content-Type header.

Access individual documents: {{.configs.defaults}}, {{.configs.custom}}, or use {{.configs}} in deepMerge to expand all.

Glob Input

Load multiple files matching a pattern:

node:
  configs:
    from_glob: "configs/*.json"

Important: Glob patterns must include a file extension.

Documents are named by the filename stem (without extension). Files are sorted lexically.

Command Input

Load configuration from command output:

node:
  # Array format (direct execution)
  repo:
    from_cmd: ["gh", "repo", "view", "--json", "owner,name,url"]

  # String format (executed via shell)
  context:
    from_cmd: "kubectl config view --output json"

  # Multiple commands
  multi:
    from_cmd:
      - ["echo", "{\"a\": 1}"]
      - ["echo", "{\"b\": 2}"]

  # Named commands
  info:
    from_cmd:
      - git: ["git", "rev-parse", "HEAD"]
      - date: ["date", "+%Y-%m-%d"]

The command output is auto-detected as JSON, YAML, or TOML.

Behavior:

  • Non-zero exit code causes an error
  • Empty output causes an error
  • Command is executed once at startup (cached like URL inputs)

Inline Input (from_expr)

Embed data directly in the panconf config:

node:
  # Single inline data (document named "doc")
  defaults:
    from_expr:
      name: myapp
      settings:
        debug: false

  # Multiple inline expressions
  configs:
    from_expr:
      - base:
          timeout: 30
          retries: 3
      - overrides:
          timeout: 60

Output

Output File

Write to a file (format inferred from extension):

node:
  result:
    to_file: output.json      # JSON format
    # to_file: output.yaml    # YAML format
    # to_file: output.toml    # TOML format
    transform:
      deepMerge: ["{{.source}}"]

Output to Stdout

Use - to write to stdout:

node:
  result:
    to_file: "-"
    transform:
      deepMerge: ["{{.source}}"]

Multiple Outputs

There are two ways to generate multiple output files:

Option 1: Multiple transform nodes

node:
  data:
    from_file: data.json
  json:
    to_file: config.json
    transform:
      deepMerge: ["{{.data}}"]
  yaml:
    to_file: config.yaml
    transform:
      deepMerge: ["{{.data}}"]

Option 2: to_file array (single transform, multiple outputs)

node:
  data:
    from_file: data.json
  out:
    to_file:
      - "{{.data}}": output.json      # Write data to JSON
      - "{{.data}}": output.yaml      # Write data to YAML
      - "{{.data}}": output.toml      # Write data to TOML
    transform:
      deepMerge: ["{{.data}}"]

The to_file array syntax uses document references as keys and output paths as values. You can reference:

  • The current transform's output: "{{.thisNode}}"
  • Input nodes: "{{.inputNode}}"
  • Specific documents from multi-doc nodes: "{{.multiDoc.docName}}"

Example: Write multiple documents from a glob node

node:
  configs:
    from_glob: "configs/*.json"    # database.json, cache.json
  out:
    to_file:
      - "{{.configs.database}}": db-config.json
      - "{{.configs.cache}}": cache-config.json
    transform:
      deepMerge: ["{{.configs}}"]

Intermediate Nodes

Transform nodes without a to_file: are intermediate - they're computed but not written to disk. They can be referenced by other nodes:

node:
  base:
    from_file: base.yaml
  env:
    from_file: env.yaml

  # Intermediate: merge base configs
  merged:
    transform:
      deepMerge:
        - "{{.base}}"
        - "{{.env}}"

  # Final: uses intermediate node
  config:
    to_file: config.json
    transform:
      deepMerge:
        - "{{.merged}}"

Nodes are automatically processed in dependency order (topological sort).

Transforms

Transform Description Docs
deepMerge Deep recursive merge, type conflicts allowed details
treeMerge Strict deep merge, errors on type conflicts details
gotmpl Go template from file with Sprig functions details
gotmpl_inline Inline Go template details
jq jq expression for data transformation details
jsonnet Jsonnet expression with std library details
cmd External command (jsonnet, jq, etc.) details 
transform:
  deepMerge: ["{{.base}}", "{{.override}}"]
  # or
  treeMerge: ["{{.base}}", "{{.override}}"]
  # or
  gotmpl: ["{{.meta.template.path}}", "{{.meta.vars.path}}"]
  # or
  gotmpl_inline:
    tmpl: |
      config:
      {{ .data | toYaml | nindent 2 }}
  # or
  jq: |
    {
      name: $config.name,
      env: $env.environment
    }
  # or
  jsonnet: |
    std.mergePatch(config, override)
  # or
  cmd: ["jsonnet", "{{.meta.template.path}}"]

See the individual transform documentation for detailed examples and options.

Template Variables

Transform arguments support Go template syntax with these variables:

The {{.self}} Variable

In hybrid nodes (nodes with both from_* and transform), the input data is available via {{.self}}:

node:
  config:
    from_file: base.json
    to_file: output.json
    transform:
      deepMerge:
        - "{{.self}}"
        - "{{.override}}"

For jq transforms, use $self instead.

Note: {{.self}} is only available in hybrid nodes. Using it in a transform-only node will result in an error.

Single-Document Nodes

For nodes with one document, use shorthand access:

node:
  config:
    from_file: config.json
  out:
    transform:
      deepMerge:
        - "{{.config}}"              # Data (shorthand for {{.config.doc}})

Multi-Document Nodes

For nodes with multiple documents:

node:
  inputs:
    from_file:
      - base.json
      - override.json
  out:
    transform:
      deepMerge:
        - "{{.inputs}}"              # All documents (in sorted order)
        - "{{.inputs.base}}"         # Specific document
        - "{{.inputs.override}}"     # Another specific document

Field Access

Access individual fields from documents in transform arguments and to_file references:

node:
  config:
    from_file: config.json    # {"database": {"host": "localhost"}, "cache": {...}}
  override:
    from_file: override.json
  out:
    to_file:
      - "{{.config.database}}": db.json      # Write only database section
      - "{{.config.cache}}": cache.json       # Write only cache section
    transform:
      deepMerge:
        - "{{.config.database}}"              # Merge only database section
        - "{{.override}}"

Field access supports nested paths:

transform:
  deepMerge:
    - "{{.config.services.api.database}}"     # Deeply nested field
    - "{{.override}}"

For multi-document nodes, the path after the node name is the document name, then field path:

node:
  configs:
    from_glob: "configs/*.json"    # base.json, prod.json
  out:
    transform:
      deepMerge:
        - "{{.configs.base.database}}"        # 'database' field in base doc
        - "{{.configs.prod.database}}"        # 'database' field in prod doc

Error handling:

  • Missing field: "field 'nonexistent' not found"
  • Non-object value in merge: "field 'port' is not an object (got float64), cannot use in merge transform"

jq Access

In jq expressions, nodes are available as variables:

node:
  config:
    from_file: config.json        # Single-doc
  inputs:
    from_file:
      - base.json                 # Multi-doc
      - override.json
  out:
    transform:
      jq: |
        {
          # Single-doc: direct access
          name: $config.name,
          
          # Multi-doc: access by doc name
          base: $inputs.base,
          override: $inputs.override
        }

Metadata Access

Access file paths and other metadata via .meta:

node:
  template:
    from_file: config.gotmpl
  data:
    from_file: vars.yaml
  out:
    transform:
      gotmpl:
        - "{{.meta.template.path}}"
        - "{{.meta.data.path}}"

Available metadata:

Property Description
.meta.node.path File path (empty for from_expr, from_url, from_cmd)
.meta.node.format Format: json, yaml, toml, jsonnet, etc.
.meta.node.name Document name

For multi-doc nodes: .meta.node.docName.path, .meta.node.docName.format, etc.

Note: Transform nodes have no metadata.

Environment Variables

Environment variables are expanded in all string values:

node:
  config:
    to_file: $HOME/config.json
    # or: ${HOME}/config.json

Tilde Expansion

~ is expanded to the user's home directory:

node:
  config:
    to_file: ~/config.json

CLI Commands

Init Command

Create a default panconf.json5 configuration file with helpful comments:

panconf init

Run Command

Generate configuration files once:

panconf run

Watch Command

Watch local input files and regenerate on changes:

panconf watch

Behavior:

  • Local files: watched for changes (modifications, deletions)
  • Glob patterns: watched for changes (new files, modifications, deletions)
  • URL inputs: downloaded once at startup and cached (not re-fetched)

Example output:

watching /path/to/base.json (node: base) watching /path/to/override.json (node: override) watching for changes... (press Ctrl+C to stop) wrote /path/to/config.json change detected: /path/to/override.json wrote /path/to/config.json

Transform Subcommands

Run transforms directly on files (useful for debugging):

# Deep merge files (output to stdout as YAML)
panconf transform deepMerge file1.json file2.yaml file3.toml

# Tree merge files (output to stdout as YAML)
panconf transform treeMerge file1.json file2.yaml

Examples

Environment-Specific Configuration

node:
  base:
    from_file: config/base.yaml
  env:
    from_file: config/$ENVIRONMENT.yaml
  config:
    to_file: config.json
    transform:
      deepMerge:
        - "{{.base}}"
        - "{{.env}}"

Run with:

ENVIRONMENT=production panconf run

Multi-File Node with Merge

node:
  configs:
    from_file:
      - defaults.yaml
      - environment.yaml
      - local.yaml
  merged:
    to_file: config.json
    transform:
      deepMerge:
        - "{{.configs}}"        # Expands to all in sorted order

Glob with Individual Document Access

node:
  services:
    from_glob: "services/*.yaml"
  manifest:
    to_file: manifest.yaml
    transform:
      gotmpl_inline:
        tmpl: |
          services:
            api:
          {{ .services.api | toYaml | nindent 4 }}
            worker:
          {{ .services.worker | toYaml | nindent 4 }}

Using Jsonnet

node:
  template:
    from_file: config.jsonnet
  config:
    to_file: config.json
    transform:
      cmd:
        - jsonnet
        - "{{.meta.template.path}}"

Cross-Format Conversion

node:
  source:
    from_file: config.yaml
  json:
    to_file: config.json
    transform:
      deepMerge:
        - "{{.source}}"
  toml:
    to_file: config.toml
    transform:
      deepMerge:
        - "{{.source}}"

Fetching Remote Config

node:
  remote:
    from_url: https://api.example.com/config.json
  local:
    from_file: local-overrides.json
  config:
    to_file: config.json
    transform:
      deepMerge:
        - "{{.remote}}"
        - "{{.local}}"

Using Command Output

node:
  base:
    from_file: base.json
  repo:
    from_cmd: ["gh", "repo", "view", "--json", "owner,name"]
  config:
    to_file: config.json
    transform:
      deepMerge:
        - "{{.base}}"
        - "{{.repo}}"

Multiple Commands with Names

node:
  info:
    from_cmd:
      - git: ["git", "rev-parse", "--short", "HEAD"]
      - date: ["date", "-u", "+%Y-%m-%dT%H:%M:%SZ"]
  build:
    to_file: build-info.json
    transform:
      jq: |
        {
          commit: $info.git,
          timestamp: $info.date
        }

Using Inline Templates

node:
  database:
    from_file: database.yaml
  cache:
    from_file: cache.yaml
  config:
    to_file: config.yaml
    transform:
      gotmpl_inline:
        tmpl: |
          services:
            database:
          {{ .database | toYaml | nindent 4 }}
            cache:
          {{ .cache | toYaml | nindent 4 }}

Using jq for Transformations

node:
  config:
    from_file: config.json
  env:
    from_file: env.yaml
  result:
    to_file: result.json
    transform:
      jq: |
        {
          name: $config.name,
          env: $env.environment,
          merged: ($config * $env)
        }

Using Jsonnet as Config File

Use panconf.jsonnet for programmable configuration with functions and imports:

// panconf.jsonnet
local env = std.extVar("ENVIRONMENT");

// Helper functions
local inputNode(file) = { from_file: file };
local mergeNode(file, sources) = {
  to_file: file,
  transform: { deepMerge: sources },
};

// Generate nodes dynamically
local environments = ["dev", "staging", "prod"];

{
  node: {
    base: inputNode("config/base.json"),
    [e]: inputNode("config/" + e + ".json") for e in environments,
    out: mergeNode("config.json", ["{{.base}}", "{{." + env + "}}"]),
  },
}

Run with:

ENVIRONMENT=prod panconf run

Validation Rules

panconf validates configuration and provides helpful error messages:

Rule Error Message
Node must be input or transform node must have either a source (from_*) or a transform
Input cannot have multiple sources must specify only one of from_file, from_url, from_glob, from_cmd, or from_expr
Glob must include extension glob pattern must include a file extension
Transform must be specified transform must specify one of cmd, deepMerge, treeMerge, gotmpl, gotmpl_inline, jq, or jsonnet
Only one transform allowed transform must specify only one of cmd, deepMerge, treeMerge, gotmpl, gotmpl_inline, jq, or jsonnet
Valid output extension (for merge transforms) file extension must be .json, .yaml, .yml, or .toml
Cycle in node dependencies cycle detected: a -> b -> a

Error Handling

Type Conflicts (treeMerge only)

When using treeMerge, type conflicts produce detailed errors:

config error: type conflict at 'settings.debug' - defined as scalar in base.json - defined as object in override.json

Command Failures

When a cmd transform fails, panconf shows the error and stderr:

command failed: exit status 1 stderr: jsonnet: parse error: ...

Missing Files

node 'config': file not found: config.json

Invalid Glob

node 'configs': glob pattern must include a file extension (e.g., '**.json')
ON THIS PAGE