Developer Guide

• Bareos Developer Notes
  • History
  • Developer Tools
  • Contributions
  • Patches
  • Issue Database
    • Working with labels
    • Reporting security issues
  • Documenting decisions
  • Memory Leaks
  • Guiding Principles
  • Boy Scout Rule
  • Usage of C++ Exceptions
  • Automatic Sourcecode Formatting
  • Formatting exceptions
  • Sourcecode Comments
  • SQL queries
  • Do’s
  • Don’ts
• Build And Test Bareos
  • Build the documentation
  • Systemtests
    • Prepare the PostgreSQL database for systemtests
    • Build Bareos for Systemtests
    • Running Systemtests
      • Run all system tests
      • Run a single system test
      • Add a systemtest
      • Directory Structures
        • Directory Structure (Source)
        • Directory Structure (Build)
  • Hints for Building Bareos
    • Configure (cmake) build settings
    • Using ccache (compiler cache)
  • Benchmarks
    • Adding a benchmark
    • Building the benchmarks
    • Running benchmarks
    • Comparing benchmarks
  • Test Coverage Analysis
    • Prerequisites
    • Building with coverage support
    • Collecting coverage information
    • Plotting analysis data
    • A complete example
• Automatic Version Generation
  • What is this about?
  • How does it work?
    • CMake Variable
    • BareosVersion.cmake
      • Generating BareosVersion.cmake
    • Git Metadata
  • Troubleshooting
    • No git
    • Not a git repository
    • Git repository missing tags
    • Building in the wrong directory
    • Last resort
• Releasing Bareos
  • Deciding for a version number
  • Preparing the release notes
    • How does it work in detail?
  • Update version-dependent files
    • version.map.in
  • Prepare the git commits and tags
    • Special considerations for major versions
  • Publishing the release
  • Updating GitHub Release
• Git Workflow
  • Getting new code or changes into Bareos
  • Releases and Backporting
  • Feature Branches
  • Commits
    • Commit message guideline
  • CHANGELOG
• Message Classes
  • Memory Messages
  • Debug Messages
  • Job Messages
    • Special Cases
  • Queued Job Messages
  • Error Messages
• Bareos FD Plugin API
  • Normal vs Command vs Options Plugins
  • Loading Plugins
  • loadPlugin
  • Plugin Entry Points
    • newPlugin(bpContext *ctx)
    • freePlugin(bpContext *ctx)
    • getPluginValue(bpContext *ctx, pVariable var, void *value)
    • setPluginValue(bpContext *ctx, pVariable var, void *value)
    • handlePluginEvent(bpContext *ctx, bEvent *event, void *value)
    • startBackupFile(bpContext *ctx, struct save_pkt *sp)
    • endBackupFile(bpContext *ctx)
    • startRestoreFile(bpContext *ctx, const char *cmd)
    • createFile(bpContext *ctx, struct restore_pkt *rp)
    • setFileAttributes(bpContext *ctx, struct restore_pkt *rp)
    • endRestoreFile(bpContext *ctx)
    • pluginIO(bpContext *ctx, struct io_pkt *io)
    • bool checkFile(bpContext *ctx, char *fname)
  • Bareos Plugin Entrypoints
    • bRC registerBareosEvents(bpContext *ctx, …)
    • bRC getBareosValue(bpContext *ctx, bVariable var, void *value)
    • bRC setBareosValue(bpContext *ctx, bVariable var, void *value)
    • bRC JobMessage(bpContext *ctx, const char *file, int line, int type, utime_t mtime, const char *fmt, …)
    • bRC DebugMessage(bpContext *ctx, const char *file, int line, int level, const char *fmt, …)
    • void bareosMalloc(bpContext *ctx, const char *file, int line, size_t size)
    • void bareosFree(bpContext *ctx, const char *file, int line, void *mem)
• Python Plugin API
  • Description of the Bareos Python plugin API for Bareos >= 20
  • Direct I/O option in Python plugin API for Bareos >= 22
  • Using large lists may cause performance issues
  • Description of the Bareos Python plugin API for Bareos < 20
  • Porting existing Python plugins to Bareos >= 20
• Platform Support
  • General
  • Requirements to become a Supported Platform
• Daemon Protocol
  • General
  • Low Level Network Protocol
  • General Daemon Protocol
  • The Protocol Used Between the Director and the Storage Daemon
  • The Protocol Used Between the Director and the File Daemon
  • The Save Protocol Between the File Daemon and the Storage Daemon
    • Command and Control Information
    • Data Information
• File Services Daemon
  • Commands Received from the Director for a Backup
  • Commands Received from the Director for a Restore
• Storage Daemon Design
  • SD Design Introduction
  • SD Development Outline
  • SD Connections and Sessions
    • SD Append Requests
    • SD Read Requests
  • SD Data Structures
• Catalog Services
  • General
    • Filenames and Maximum Filename Length
    • Database Table Design
  • Database Tables
    • Path
    • File
    • Job
    • FileSet
    • JobMedia
    • Device
      • Tape Volume
      • Other Volume
    • Media (Volume)
    • Pool
    • Client
    • Storage
    • Counters
    • Log
    • Location
    • LocationLog
    • Version
    • BaseFiles
  • UML Diagram of Database Schema
• Configuration Parser
  • Introduction
  • Class Model of Storage Daemon Resources
• Storage Media Output Format
  • General
  • Definitions
  • Overall Format
    • Storage Daemon File Output Format
  • Serialization
  • Block Header
  • Record Header
  • Volume Label Format
  • Session Label
  • Overall Storage Format
  • Examine Volumes
    • bls command
  • Unix File Attributes
• Bareos Porting Notes
  • Porting Requirements
• API
  • General
    • Minimal Code in Console Program
    • GUI Interface is Difficult
  • dot commands
  • API Modes
    • API mode 0 (off)
    • API mode 1 (on)
    • API mode 2 (json)
      • Examples
        • Example of a JSON-RPC Error Response
  • Bvfs API
    • General notes
    • Get dependent jobs from a given JobId
    • Generating Bvfs cache
    • List directories
    • List files
      • API mode 0
      • API mode 1
      • API mode 2
    • Get all versions of a specific file
    • Restore set of files
    • Cleanup after Restore
    • Clearing the BVFS Cache
    • Example for directory browsing using bvfs
  • REST API
    • Introduction
    • Installation
    • Configuration
    • Start the backend server
    • Browse
    • Future work
• TLS
  • Introduction
  • TLS Handshake before Bareos 18.2
  • TLS Configuration Implementation
  • TLS API Implementation
• PAM
• Tests
  • Unit Tests
    • Adding a new C++ Test
    • Adding tests in general
• Bareos Memory Management
  • General
    • Statically Allocated Memory
    • Dynamically Allocated Memory
    • Pooled and Non-pooled Memory
      • Types of Memory Pool:
      • Getting Memory:
      • Releasing Memory:
      • Determining the Memory Size:
      • Resizing Pool Memory:
      • Automatic Size Adjustment:
      • Releasing All Pooled Memory:
      • Pooled Memory Statistics:
• TCP/IP Network Protocol
  • Introduction
  • Network Routines
    • bnet and Threads
    • bnet_open
    • bnet_send
    • bnet_fsend
    • is_bnet_error
    • is_bnet_stop
    • bnet_recv
    • bnet_sig
    • bnet_strerror
    • bnet_close
    • Becoming a Server
  • Higher Level Conventions
  • Network startup activity
  • Network Sequence Diagrams
    • Backup of a Single File
      • Overview
      • Detailed Diagram
    • Restore of a Single File
      • Overview single file restore
      • Detailed Diagram single file restore
    • Migration of a Single Volume
      • Overview single volume migration
      • Detailed Diagram single volume migration
    • Verify of a Single File (Mode: Volume to Catalog)
      • Overview single file verify
      • Detailed Diagram single file verify
• Director Console Output
  • object_key_value
  • decoration
  • messages
  • Objects and Arrays
    • named objects
    • nameless objects
    • arrays
  • Example
  • Example with 3 level structure
• Reservation
  • Introduction
    • Diagram Conventions
  • UseDeviceCmd
  • ReserveDevice
• Dplcompat Wrapper Programs
  • Overview
  • Operations
    • options operation
    • testconnection operation
    • list operation
    • stat operation
    • upload operation
    • download operation
    • remove operation
  • Minimum viable example
• Dedupable Backend
  • Block Level Deduping Filesystems
  • Storing data in a dedupable way
    • So how does this look in practice:
  • Volume Structure
• Job Execution
  • Introduction
  • Job setup and start
  • Simple backup job
    • Overview simple backup job
    • Detailed View simple backup job
  • Simple restore job
    • Overview simple restore job
    • Detailed View simple restore job
  • Local copy or migrate job
    • Overview local copy job
    • Detailed View local copy job
  • Remote copy or migrate job
    • Overview remote copy job
    • Detailed View remote copy job
• Python
  • Automatic Sourcecode Formatting
  • Docstrings
• python-bareos Module
  • bareos
  • bareos.bsock
    • Preparations
      • Calling bareos-director console commands
      • Simple version of the bconsole in Python
      • Use JSON objects of the API mode 2
    • Transport Encryption (TLS-PSK)
      • Fallback To Unencrypted Connections
      • sslpsk
  • bareos.bsock.connectiontype
    • bareos.bsock.connectiontype.ConnectionType
      • ConnectionType
  • bareos.bsock.constants
    • bareos.bsock.constants.Constants
      • Constants
        • Constants.get_description()
  • bareos.bsock.directorconsole
    • bareos.bsock.directorconsole.DirectorConsole
      • DirectorConsole
        • DirectorConsole.argparser_add_default_command_line_arguments()
        • DirectorConsole.argparser_get_bareos_parameter()
        • DirectorConsole.auth()
        • DirectorConsole.call()
        • DirectorConsole.close()
        • DirectorConsole.connect()
        • DirectorConsole.get_cipher()
        • DirectorConsole.get_protocol_version()
        • DirectorConsole.get_tls_psk_identity()
        • DirectorConsole.has_data()
        • DirectorConsole.interactive()
        • DirectorConsole.is_connected()
        • DirectorConsole.is_end_of_message()
        • DirectorConsole.is_tls_psk_available()
        • DirectorConsole.receive_and_evaluate_response_message()
        • DirectorConsole.reconnect()
        • DirectorConsole.recv()
        • DirectorConsole.recv_bytes()
        • DirectorConsole.recv_msg()
        • DirectorConsole.recv_submsg()
        • DirectorConsole.send()
        • DirectorConsole.send_command()
  • bareos.bsock.directorconsolejson
    • bareos.bsock.directorconsolejson.DirectorConsoleJson
      • DirectorConsoleJson
        • DirectorConsoleJson.argparser_add_default_command_line_arguments()
        • DirectorConsoleJson.argparser_get_bareos_parameter()
        • DirectorConsoleJson.auth()
        • DirectorConsoleJson.call()
        • DirectorConsoleJson.call_fullresult()
        • DirectorConsoleJson.close()
        • DirectorConsoleJson.connect()
        • DirectorConsoleJson.get_cipher()
        • DirectorConsoleJson.get_protocol_version()
        • DirectorConsoleJson.get_tls_psk_identity()
        • DirectorConsoleJson.has_data()
        • DirectorConsoleJson.interactive()
        • DirectorConsoleJson.is_connected()
        • DirectorConsoleJson.is_end_of_message()
        • DirectorConsoleJson.is_tls_psk_available()
        • DirectorConsoleJson.receive_and_evaluate_response_message()
        • DirectorConsoleJson.reconnect()
        • DirectorConsoleJson.recv()
        • DirectorConsoleJson.recv_bytes()
        • DirectorConsoleJson.recv_msg()
        • DirectorConsoleJson.recv_submsg()
        • DirectorConsoleJson.send()
        • DirectorConsoleJson.send_command()
  • bareos.bsock.filedaemon
    • bareos.bsock.filedaemon.FileDaemon
      • FileDaemon
        • FileDaemon.argparser_add_default_command_line_arguments()
        • FileDaemon.argparser_get_bareos_parameter()
        • FileDaemon.auth()
        • FileDaemon.call()
        • FileDaemon.close()
        • FileDaemon.connect()
        • FileDaemon.get_cipher()
        • FileDaemon.get_protocol_version()
        • FileDaemon.get_tls_psk_identity()
        • FileDaemon.has_data()
        • FileDaemon.interactive()
        • FileDaemon.is_connected()
        • FileDaemon.is_end_of_message()
        • FileDaemon.is_tls_psk_available()
        • FileDaemon.receive_and_evaluate_response_message()
        • FileDaemon.reconnect()
        • FileDaemon.recv()
        • FileDaemon.recv_bytes()
        • FileDaemon.recv_msg()
        • FileDaemon.recv_submsg()
        • FileDaemon.send()
        • FileDaemon.send_command()
  • bareos.bsock.lowlevel
    • bareos.bsock.lowlevel.format_warning_short
      • format_warning_short()
    • bareos.bsock.lowlevel.LowLevel
      • LowLevel
        • LowLevel.argparser_get_bareos_parameter()
        • LowLevel.auth()
        • LowLevel.call()
        • LowLevel.close()
        • LowLevel.connect()
        • LowLevel.get_cipher()
        • LowLevel.get_protocol_version()
        • LowLevel.get_tls_psk_identity()
        • LowLevel.has_data()
        • LowLevel.interactive()
        • LowLevel.is_connected()
        • LowLevel.is_end_of_message()
        • LowLevel.is_tls_psk_available()
        • LowLevel.receive_and_evaluate_response_message()
        • LowLevel.reconnect()
        • LowLevel.recv()
        • LowLevel.recv_bytes()
        • LowLevel.recv_msg()
        • LowLevel.recv_submsg()
        • LowLevel.send()
        • LowLevel.send_command()
  • bareos.bsock.protocolmessageids
    • bareos.bsock.protocolmessageids.ProtocolMessageIds
      • ProtocolMessageIds
  • bareos.bsock.protocolmessages
    • bareos.bsock.protocolmessages.ProtocolMessages
      • ProtocolMessages
        • ProtocolMessages.pam_user_credentials()
  • bareos.bsock.protocolversions
    • bareos.bsock.protocolversions.ProtocolVersions
      • ProtocolVersions
  • bareos.bsock.tlsversionparser
    • bareos.bsock.tlsversionparser.ArgParserTlsVersionAction
      • ArgParserTlsVersionAction
        • ArgParserTlsVersionAction.__call__()
    • bareos.bsock.tlsversionparser.TlsVersionParser
      • TlsVersionParser
  • bareos.util
  • bareos.util.bareosbase64
    • bareos.util.bareosbase64.BareosBase64
      • BareosBase64
        • BareosBase64.base64_to_int()
        • BareosBase64.int_to_base64()
        • BareosBase64.string_to_base64()
        • BareosBase64.twos_comp()
  • bareos.util.password
    • bareos.util.password.Password
      • Password
        • Password.md5()
        • Password.plaintext()
        • Password.set_md5()
        • Password.set_plaintext()
  • bareos.util.path
    • bareos.util.path.Path
      • Path
        • Path.shift()
  • bareos.exceptions
    • bareos.exceptions.AuthenticationError
      • AuthenticationError
    • bareos.exceptions.ConnectionError
      • ConnectionError
    • bareos.exceptions.ConnectionLostError
      • ConnectionLostError
    • bareos.exceptions.Error
      • Error
    • bareos.exceptions.JsonRpcErrorReceivedException
      • JsonRpcErrorReceivedException
    • bareos.exceptions.JsonRpcInvalidJsonReceivedException
      • JsonRpcInvalidJsonReceivedException
    • bareos.exceptions.PamAuthenticationError
      • PamAuthenticationError
    • bareos.exceptions.SignalReceivedException
      • SignalReceivedException
    • bareos.exceptions.SocketEmptyHeader
      • SocketEmptyHeader
• PHP
  • Guiding Principles
  • Automatic Sourcecode Formatting
• Webui
  • Translation
    • Where do I find the translation?
    • How should I edit translation files?
    • How can I add a new language?
    • Where can I find further information?
    • How to update files in module/Application/language if there were changes in source that affect translation?
  • Command usage in modules and the according ACL requirements
• Developer FAQ
  • How to run a job that automatically fails?
  • I have changes that affect tapes and autochangers, how can I test them?