SwarmKit Architecture¶
Author : Rajdeep Dua
Last Updated : June 20 2016
Introduction¶
In this article we look at the overall Architecture of Swarmkit. Swarmkit is a distributed resource manager. This can be bundled to run Docker tasks or other types of Tasks.
Main components of Swarmkit:
Swarmkit is composed two types of Nodes
- Managers : Responsible for Assigning Tasks to Workers
- Workers : Place where actual tasks run
Following entities are scheduled on Workerss
- Tasks is the unit of work performed by a Worker.
- Service is a bundle of Tasks which are run as a single unit and monitored by the Manager
Node¶
Node struct implements the node functionality for a member of a swarm cluster. Node handles workloads (as a worker) and may also run as a manager.
type Node struct {
sync.RWMutex
config *NodeConfig
remotes *persistentRemotes
role string
conn *grpc.ClientConn
connCond *sync.Cond
nodeID string
....
err error
agent *Agent
manager *manager.Manager
roleChangeReq chan api.NodeRole
}
Every node has a reference to a Manager instance and Agent instance.
Node can have four states
- started
- stopped
- ready
- closed
Manager¶
Manager has following components at Runtime
- Dispatcher
- Allocator
- Scheduler
- GRPC Server
- GRPC Local Server
- Global Orchestrator

golang struct for the manager is listed below
type Manager struct {
config *Config
listeners map[string]net.Listener
caserver *ca.Server
Dispatcher *dispatcher.Dispatcher
replicatedOrchestrator *orchestrator.ReplicatedOrchestrator
globalOrchestrator *orchestrator.GlobalOrchestrator
taskReaper *orchestrator.TaskReaper
scheduler *scheduler.Scheduler
allocator *allocator.Allocator
keyManager *keymanager.KeyManager
server *grpc.Server
localserver *grpc.Server
RaftNode *raft.Node
}
Each Manager listens on a IP Address and port for GRPC messages. Two separate RPC servers run for listening to local and remote connections.
Dispatcher¶
Dispatcher is responsible for dispatching tasks and tracking agent health.
It has reference to the following elements.
List of managers
lastSeenManagers []*api.WeightedPeer This is the last where tasks are dispatchedRaft cluster :Cluster from where dispatcher gets list of managers
Manager Queue where messages are published
Each dispatcher has reference to list of Nodes
Nodes are registered in the dispatcher using following call
func (d *Dispatcher) register(ctx context.Context, nodeID string, description *api.NodeDescription) (string, string, error) { ... rn := d.nodes.Add(node, expireFunc) ... }Dispatches Tasks to Nodes
Scheduler¶
Assigns tasks to the Nodes. Following datastructure represents a scheduler. It has reference to following tasks
- UnAssigned Tasks
unassignedTasks
- PreAssigned Tasks
preassignedTasks
- All Tasks
allTasks
type Scheduler struct {
store *store.MemoryStore
unassignedTasks *list.List
preassignedTasks map[string]*api.Task
nodeHeap nodeHeap
allTasks map[string]*api.Task
}
Agent¶
Agent implements primary node functionality for a member of a swarm cluster. The primary functionality of this Agent is to run on the Node and provide status of the tasks.

Components of a Agent instance are
- Reporter
- Controller
- Task
- Worker
- Storage
- Node
- exec.Controller
- exec.container.Controller
Agent is represented by the following datastructure
type Agent struct {
config *Config
node *api.Node
keys []*api.EncryptionKey
sessionq chan sessionOperation
worker Worker
...
}
Worker¶
Worker implements the core task management logic and persistence. It coordinates the set of assignments with the executor. Each Agent has a single worker which has reference to taskManager Map: map[string]*taskManager
and Executor : exec.Executor
. Each worker also has a set of Listeners ( map[*statusReporterKey]struct{}
) giving status of the tasks.
type worker struct {
db *bolt.DB
executor exec.Executor
listeners map[*statusReporterKey]struct{}
taskManagers map[string]*taskManager
}
TaskManager¶
Manages all the tasks for an agent. Has reference to exec.Controller
type taskManager struct {
task *api.Task
ctlr exec.Controller
reporter StatusReporter
updateq chan *api.Task
shutdown chan struct{}
closed chan struct{}
}
Reporter¶
Reports status of the Tasks. Maintains a map of task ids and the Task Object
type statusReporter struct {
reporter StatusReporter
statuses map[string]*api.TaskStatus
...
}
Controller¶
Interface for managing lifecycle of a Task. Makes the call to underlying implementation which talks to Docker APIs
Update(ctx context.Context, t *api.Task) error
Prepare(ctx context.Context) error
Start(ctx context.Context) error
Wait(ctx context.Context) error
Shutdown(ctx context.Context) error
Terminate(ctx context.Context) error
Remove(ctx context.Context) error
Close() error
ContainerConfig¶
Reference to the underlying Docker APIs for making the actual calls