🍳 Chef Guide

Chef: recipes, cookbooks, attributes and the client/server model
cinc-apply web.rb --why-run

Chef is a configuration management tool where the configuration is Ruby code. You write recipes (lists of resources: this package, this file, this service), group them into cookbooks, and a client on each machine converges the machine to what the recipes say, over and over. This page builds a small cookbook from nothing and then runs it every way Chef offers: as a single file, from a repository in "local mode", and from a server with registered nodes, roles, environments, data bags and Policyfiles. It also shows the places where Chef does something other than what the recipe seems to say: Ruby code that runs before any resource does, a reload that is silently a no-op, a notification that fails because another recipe is missing, a "server" that checks no passwords at all.
A word on names, because it matters for honesty. I used CINC, the community build of Chef Infra: packages (the package metadata says License: Apache-2.0) of the same Chef Infra Client code (the Ruby gem inside the package is chef-19.3.14), with the commands renamed: cinc-client for chef-client, cinc-apply for chef-apply, cinc-auditor for inspec and so on (the package also installs the chef-client, chef-apply and chef-shell names; knife and cookstyle keep their names). Chef's own packages come with a license that has to be accepted; I did not install them, and I did not accept any license on anybody's behalf. The recipes and the cookbook on this page are plain Chef code that I wrote for it; I did not test other people's cookbooks (for example from the Supermarket) with Cinc. This page is a tour of what I ran, so every command here is the Cinc spelling. Where Chef's own tools would differ (config directories, version numbers), the page says "not run".
Every output below ran for real, on 9 October 2026: Cinc Client 19.3.14 (Ohai 19.1.31, Ruby 3.4.9) and Cinc Workstation 26.3.0 (knife 19.3.3, Cookstyle 9.0.0, Cinc Auditor 7.0.107, cinc-zero 15.1.11), every package checked against the checksum published by the Cinc download service, on Ubuntu 24.04 Docker containers. Containers are not servers: there is no systemd inside, so Chef used its init-script service provider, and that matters in two places below. Where something was not run, this page says "not run in the lab".

Docker containers on a private network, all built from one image (ubuntu:24.04 plus curl, sudo, git and a few tools), started with docker run --init:
  ws      the workstation: Cinc Workstation 26.3.0 (it also contains the client), the cookbook repository, knife, and a cinc-zero server
  node1   a machine with only the client (Cinc Client 19.3.14)
  node2   another one
$ cinc-client --version; ohai --version
Cinc Client: 19.3.14
Ohai: 19.1.31
Two harmless things in the outputs, removed from this page to keep it readable. Every run in these containers prints:
ERROR: shard_seed: Failed to get dmi property serial_number: is dmidecode installed?
(a container has no hardware serial number to read), and the workstation prints "Please install an English UTF-8 locale for Cinc Client to use, falling back to C locale and disabling UTF-8 support." Both are only noise here. The second one stops being noise for the Cinc Server package, whose cinc-server-ctl command crashed with "invalid multibyte char (US-ASCII)" until I installed a UTF-8 locale (see "What was not run").
The --init matters for the same reason as in the Puppet page of this site: a container's process 1 must reap dead child processes, or a stopped daemon stays as a zombie and an init-script stop waits for it forever.

  resource    one thing on the machine, written as a Ruby method call with a block: package 'nginx', file '/tmp/x' do ... end, service 'nginx' do ... end. Each has an ACTION (:install, :create, :delete, :start...) and properties
  recipe      a Ruby file of resources, run top to bottom
  cookbook    a directory of recipes, attributes, templates, files and custom resources, with a metadata.rb
  run list    which recipes (or roles) a node runs, in order: recipe[webserver], role[web]
  node        a machine with its attributes: the data about it (from Ohai) and the data you give it (from cookbooks, roles, environments)
  Infra Phase one run of the client. It has two stages, and the second one is the one people forget about: first it READS the recipes and builds a list of resources (compile), then it applies that list (converge). See below
Like the other tools, Chef compares each resource with the machine and changes only what differs; the output marks the unchanged ones "(up to date)". The client can run alone on one machine (cinc-apply, or cinc-client in local mode, "-z") or talk to a Chef server that stores the cookbooks, roles, node objects and data.

Ohai is Chef's fact collector, and its output is the "automatic" attributes of the node:
$ ohai platform platform_version platform_family kernel/release
[
  "ubuntu"
]
[
  "24.04"
]
[
  "debian"
]
[
  "7.0.0-38-generic"
]
$ ohai hostname fqdn
[
  "node1"
]
[
  "node1.lab"
]
$ ohai init_package
[
  "docker-init"
]
(the last one is why the service provider below is the init-script one: in this container "process 1" is Docker's init, not systemd.) In a recipe these facts are node['platform'], node['hostname'], node['cpu']['total'] and so on.

cinc-apply runs one recipe file, with no cookbook, no server, nothing else. This one installs nginx, removes its default site, adds a site, and keeps the service running:
$ cat web.rb
# A tiny web server: package, config, content, service
apt_update 'refresh' do
  frequency 86400
  action :periodic
end

package 'nginx'

file '/etc/nginx/sites-enabled/default' do
  action :delete
  notifies :restart, 'service[nginx]', :delayed
end

directory '/var/www/hello' do
  recursive true
end

file '/var/www/hello/index.html' do
  content "<h1>Hello from #{node['hostname']}</h1>\n"
end

file '/etc/nginx/conf.d/hello.conf' do
  content "server {\n    listen 80;\n    root /var/www/hello;\n}\n"
  notifies :restart, 'service[nginx]', :delayed
end

service 'nginx' do
  action [:enable, :start]
end
A trap before the first run: the recipe file must be the FIRST argument. Options in front of it are taken as the file name:
$ cinc-apply --no-color -W web.rb
FATAL: No file exists at /root/--no-color
(its own usage line says so: cinc-apply [RECIPE_FILE | -e RECIPE_TEXT | -s] [OPTIONS].) The preview, "why-run" (-W):
$ cinc-apply web.rb --no-color -W
Recipe: (cinc-apply cookbook)::(cinc-apply recipe)
  * apt_update[refresh] action periodic
    - Would update new lists of packages
  * apt_package[nginx] action install
    - Would install version 1.24.0-2ubuntu7.18 of package nginx
  * file[/etc/nginx/sites-enabled/default] action delete (up to date)
  * directory[/var/www/hello] action create
    - Would create new directory /var/www/hello
  * file[/var/www/hello/index.html] action create
    * Parent directory /var/www/hello does not exist.
    * Assuming directory /var/www/hello would have been created
    - Would create new file /var/www/hello/index.html
  * file[/etc/nginx/conf.d/hello.conf] action create
    * Parent directory /etc/nginx/conf.d does not exist.
    * Assuming directory /etc/nginx/conf.d would have been created
    - Would create new file /etc/nginx/conf.d/hello.conf
  * service[nginx] action enable
    - Would enable service service[nginx]
  * service[nginx] action start
    - Would start service service[nginx]
  * service[nginx] action restart
    - Would restart service service[nginx]
Why-run is a good guess, not a promise: it cannot know that installing the package would create the default site, so the delete says "(up to date)" now and the real run below does delete it. The real run (trimmed to the resource lines):
$ time cinc-apply web.rb --no-color
Recipe: (cinc-apply cookbook)::(cinc-apply recipe)
  * apt_update[refresh] action periodic
    ...
  * apt_package[nginx] action install
    - install version 1.24.0-2ubuntu7.18 of package nginx
  * file[/etc/nginx/sites-enabled/default] action delete
    - delete file /etc/nginx/sites-enabled/default
  * directory[/var/www/hello] action create
    - create new directory /var/www/hello
  * file[/var/www/hello/index.html] action create
    - create new file /var/www/hello/index.html
    - update content in file /var/www/hello/index.html from none to dc7e5f
    --- /var/www/hello/index.html	2026-10-09 13:32:29.501433434 +0000
    +++ /var/www/hello/.chef-index20261009-434-7fyorx.html	2026-10-09 13:32:29.500433440 +0000
    @@ -0,0 +1 @@
    +<h1>Hello from node1</h1>
  * file[/etc/nginx/conf.d/hello.conf] action create
    - create new file /etc/nginx/conf.d/hello.conf
    ...
  * service[nginx] action enable (up to date)
  * service[nginx] action start
    - start service service[nginx]
  * service[nginx] action restart
    - restart service service[nginx]

real	0m12.827s
Then the same command again:
$ cinc-apply web.rb --no-color
Recipe: (cinc-apply cookbook)::(cinc-apply recipe)
  * apt_update[refresh] action periodic (up to date)
  * apt_package[nginx] action install (up to date)
  * file[/etc/nginx/sites-enabled/default] action delete (up to date)
  * directory[/var/www/hello] action create (up to date)
  * file[/var/www/hello/index.html] action create (up to date)
  * file[/etc/nginx/conf.d/hello.conf] action create (up to date)
  * service[nginx] action enable (up to date)
  * service[nginx] action start (up to date)
$ curl -s localhost/
<h1>Hello from node1</h1>
Everything "(up to date)": the recipe is idempotent. Things to read in the first run: files show a unified diff (the checksum shown is the first six characters), the "--- ... +++ ..." lines show the real file against the temporary one Chef wrote next to it (.chef-NAME...), and the service was STARTED and then RESTARTED in the same run: the restart is the delayed notification from the two files, which fired even though the service had only just started. A double start is harmless here and a surprise the first time you see it.
Two warnings seen in the first run: deleting the default site, which is a symlink, printed "File /etc/nginx/sites-enabled/default managed by file[...] is really a symlink ... Disable this warning by setting `manage_symlink_source true` on the resource" (the cookbook below sets manage_symlink_source false, and it deleted the link all the same). And the exit code: cinc-apply returned 0 after a run that changed things, and 1 after a failure (see the errors section); no "detailed" exit code is needed to tell success from failure, but you also cannot tell "changed" from "nothing to do" by the exit code. The summary line of cinc-client ("Infra Phase complete, 13/16 resources updated") is what to read.

$ sed -i 's/listen 80;/listen 8080;/' /etc/nginx/conf.d/hello.conf; service nginx stop
$ cinc-apply web.rb --no-color | grep -v 'up to date'
Recipe: (cinc-apply cookbook)::(cinc-apply recipe)
  * file[/etc/nginx/conf.d/hello.conf] action create
    - update content in file /etc/nginx/conf.d/hello.conf from 3e9e11 to f2dbcb
    --- /etc/nginx/conf.d/hello.conf	2026-10-09 13:32:52.162757032 +0000
    +++ /etc/nginx/conf.d/.chef-hello20261009-1199-py2fug.conf	2026-10-09 13:32:57.606272589 +0000
    @@ -1,4 +1,4 @@
     server {
    -    listen 8080;
    +    listen 80;
         root /var/www/hello;
     }
  * service[nginx] action start
    - start service service[nginx]
  * service[nginx] action restart
    - restart service service[nginx]
$ curl -s localhost/; grep listen /etc/nginx/conf.d/hello.conf
<h1>Hello from node1</h1>
    listen 80;
Both repaired.

This is the single most important thing to understand about Chef, and it is not in the resource syntax. A recipe is Ruby code. Plain Ruby (an if, a puts, File.exist?) runs while the recipe is being READ. Resources do not run then: they are only added to a list. After all recipes are read, the list is applied. So a plain Ruby test of the state of the machine tests the state BEFORE any resource ran:
$ cat phase.rb
puts "COMPILE: /tmp/made-by-resource exists? #{File.exist?('/tmp/made-by-resource')}"

file '/tmp/made-by-resource' do
  content "x\n"
end

if File.exist?('/tmp/made-by-resource')
  log 'branch A: the file existed when the recipe was read'
else
  log 'branch B: the file did not exist when the recipe was read'
end

ruby_block 'look again' do
  block { puts "CONVERGE: /tmp/made-by-resource exists? #{File.exist?('/tmp/made-by-resource')}" }
end
$ cinc-apply phase.rb --no-color        (first run, the file does not exist)
COMPILE: /tmp/made-by-resource exists? false
Recipe: (cinc-apply cookbook)::(cinc-apply recipe)
  * file[/tmp/made-by-resource] action create
    - create new file /tmp/made-by-resource
  * log[branch B: the file did not exist when the recipe was read] action write
  * ruby_block[look again] action runCONVERGE: /tmp/made-by-resource exists? true
$ cinc-apply phase.rb --no-color        (second run)
COMPILE: /tmp/made-by-resource exists? true
  * file[/tmp/made-by-resource] action create (up to date)
  * log[branch A: the file existed when the recipe was read] action write
  * ruby_block[look again] action runCONVERGE: /tmp/made-by-resource exists? true
The first run took "branch B" even though the file exists a moment later: the if ran at compile time, before the file resource did. Only on the SECOND run does the branch change. This is the classic source of recipes that "work on the second run". The cures: put the test in a resource guard (only_if / not_if with a block or a command), which is evaluated at converge time, or inside a ruby_block (its code runs at converge time), or use the lazy evaluation that many properties allow. Plain Ruby at the top level of a recipe is fine for things that do not change during the run: node attributes, loops over data, string building.

A notification says "when this resource changes, run an action on that one":
  notifies :restart, 'service[nginx]', :delayed       run it at the END of the run (the default, and it runs once however many resources asked)
  notifies :restart, 'service[nginx]', :immediately   run it right after this resource
  subscribes :restart, 'service[nginx]'               the same relation, written on the other side
A test of the order, using ruby_blocks that write their names to a file:
$ cat notif.rb
mark = lambda do |name|
  lambda { ::File.open('/tmp/order.log', 'a') { |f| f.puts name } }
end

ruby_block 'first' do
  block(&mark.call('first'))
  notifies :run, 'ruby_block[after-first-immediately]', :immediately
end

ruby_block 'second' do
  block(&mark.call('second'))
end

ruby_block 'third' do
  block(&mark.call('third'))
  notifies :run, 'ruby_block[delayed-at-the-end]', :delayed
end

ruby_block 'after-first-immediately' do
  block(&mark.call('after-first-immediately (notified)'))
  action :nothing
end

ruby_block 'delayed-at-the-end' do
  block(&mark.call('delayed-at-the-end (notified)'))
  action :nothing
end

ruby_block 'fourth' do
  block(&mark.call('fourth'))
end
$ cat /tmp/order.log
first
after-first-immediately (notified)
second
third
fourth
delayed-at-the-end (notified)
Immediately means right after the notifying resource, delayed means after everything else. A resource with action :nothing does nothing by itself and exists only to be notified.
Two more things I ran into while writing that test. The first version used a log resource to notify from, and nothing fired: a log resource never counts as "changed", so what it notifies never runs. And my first version defined a helper with def mark(name) at the top of the recipe; inside a resource block it failed:
NoMethodError: undefined method `mark' for Chef::Resource::RubyBlock
The block runs in the context of the resource, where a def from the recipe is not visible. A local variable (the lambda above) IS visible, because a block is a closure.
Guards decide whether a resource runs at all, and they are what makes execute idempotent. Without one, execute runs on every run:
$ cat guards.rb
execute 'no guard' do
  command 'echo ran > /tmp/g-noguard'
end

execute 'creates' do
  command 'echo ran > /tmp/g-creates'
  creates '/tmp/g-creates'
end

execute 'not_if' do
  command 'echo ran > /tmp/g-notif'
  not_if 'test -f /tmp/g-notif'
end

execute 'only_if' do
  command 'echo ran > /tmp/g-onlyif'
  only_if { node['platform'] == 'ubuntu' }
  not_if { ::File.exist?('/tmp/g-onlyif') }
end
$ cinc-apply guards.rb --no-color       (second run)
  * execute[no guard] action run
    - execute echo ran > /tmp/g-noguard
  * execute[creates] action run (up to date)
  * execute[not_if] action run (skipped due to not_if)
  * execute[only_if] action run (skipped due to not_if)
The unguarded one runs every time and makes every run report an update; creates, a string guard and a block guard all keep the others quiet. A string guard is a shell command (for not_if the resource is skipped when the command succeeds); a block guard is Ruby. Prefer a real resource (package, file, service, user) to an execute whenever there is one.

This one cost me an hour, because everything looks right. The recipe notifies the service with :reload when a config file changes. Chef runs it, prints the line, and moves on:
  * service[nginx] action reload
    - reload service service[nginx]
and nothing happens: nginx keeps running the old configuration. I checked it objectively, by counting nginx's worker processes before and after (a reload replaces them), with and without the setting that fixes it:
=== WITH supports reload: true
workers before: 2444 2445 2446 2447 2448 2449 2450 2451
    - reload service service[nginx]
workers after:  2758 2759 2760 2761 2762 2763 2764 2765
=== WITHOUT it
workers before: 2758 2759 2760 2761 2762 2763 2764 2765
    - reload service service[nginx]
workers after:  2758 2759 2760 2761 2762 2763 2764 2765
The reason is in the provider. For init-script services, the code of the version I ran (Chef Infra Client 19.3.14, as Cinc) says:
def reload_service
  if @new_resource.reload_command
    super
  elsif supports[:reload]
    shell_out!("#{default_init_command} reload", default_env: false)
  end
end
so with neither a reload_command nor supports reload: true, it does NOTHING, and Chef still reports the action as done. The cookbook below therefore says:
service 'nginx' do
  supports reload: true, restart: true, status: true
  action %i(enable start)
end
Which provider you get depends on the machine: on a systemd host Chef uses its systemd provider (the file exists in the same directory), which I did not run. The lesson holds anyway: after a notification, verify the effect (a changed process, a response from the new config), not the log line.
A second thing about reloads: it is asynchronous. nginx's reload returns when the signal is sent, and the new workers take over a moment later. Polling the site twice a second right after a run finished:
start 13:43:13.153
run finished, exit=0 at 13:43:22.997
13:43:23.003  <h1>Welcome to nginx!
13:43:23.520  <h1>Welcome to the shop
13:43:24.045  <h1>Welcome to the shop
The first request, six milliseconds after Chef returned, still got the old configuration; half a second later the new one. A check that runs right after the converge needs a short retry. In the first draft of my own test I saw "the default page" and nearly published a wrong explanation.

A cookbook is the unit you share and version. The workstation can generate one:
$ cinc generate cookbook webserver
Generating cookbook webserver
- Ensuring correct cookbook content
- Committing cookbook files to git

Your cookbook is ready. Type `cd webserver` to enter it.
It created a git repository with these files (not counting .git): .gitignore, CHANGELOG.md, LICENSE, Policyfile.rb, README.md, chefignore, compliance/README.md, kitchen.yml, metadata.rb, recipes/default.rb (empty but for a header comment), and test/integration/default/default_test.rb (an example Cinc Auditor test). The Policyfile.rb it generates:
$ cat Policyfile.rb
name 'webserver'
default_source :supermarket
run_list 'webserver::default'
cookbook 'webserver', path: '.'
and the generated kitchen.yml is set up for a Vagrant driver and lists nine platforms from almalinux-9 to ubuntu-24.04; Test Kitchen was not run in the lab (it needs a VM or container driver). The generated tests are marked :skip until you edit them.
The cookbook of this page is the same shape with real content. The files:
cookbooks/webserver/metadata.rb
cookbooks/webserver/attributes/default.rb
cookbooks/webserver/recipes/default.rb
cookbooks/webserver/resources/site.rb
cookbooks/webserver/templates/site.conf.erb
cookbooks/webserver/files/robots.txt
$ cat metadata.rb
name 'webserver'
maintainer 'TechCommune Lab'
maintainer_email 'lab@example.org'
license 'Apache-2.0'
description 'Installs nginx and one virtual host per entry of node["webserver"]["sites"]'
version '0.1.0'
chef_version '>= 16.0'
supports 'ubuntu'
$ cat attributes/default.rb
default['webserver']['package'] = 'nginx'
default['webserver']['port'] = 80
default['webserver']['sites'] = {
  'shop' => 'Welcome to the shop',
  'blog' => 'Read the blog',
}
$ cat recipes/default.rb
apt_update 'refresh' do
  frequency 86_400
  action :periodic
end

package node['webserver']['package']

service 'nginx' do
  supports reload: true, restart: true, status: true
  action %i(enable start)
end

file '/etc/nginx/sites-enabled/default' do
  action :delete
  manage_symlink_source false
  notifies :reload, 'service[nginx]', :delayed
end

directory '/var/www' do
  mode '0755'
end

cookbook_file '/var/www/robots.txt' do
  source 'robots.txt'
  mode '0644'
end

node['webserver']['sites'].each do |site_name, text|
  webserver_site site_name do
    message text
    port node['webserver']['port']
  end
end
The recipe is short because the per-site work is in a custom resource, and the data (which sites, which port) is in attributes: change the data, not the recipe. cookbook_file copies a file out of the cookbook's files/ directory; template renders an ERB file out of templates/:
$ cat templates/site.conf.erb
# Managed by Chef (cookbook webserver)
server {
    listen <%= @port %>;
    server_name <%= @site %>.example.org;
    root /var/www/<%= @site %>;
}
The values come from variables(...) on the template resource, written as @name in the ERB.

A custom resource is your own resource type, defined in one file under resources/. The file name gives the name (site.rb in the webserver cookbook is webserver_site, with provides :webserver_site):
$ cat resources/site.rb
# A custom resource: one nginx virtual host, with its page
provides :webserver_site
unified_mode true

property :site_name, String, name_property: true
property :message, String, required: true
property :port, Integer, default: 80

action :create do
  directory "/var/www/#{new_resource.site_name}" do
    mode '0755'
  end

  template "/etc/nginx/conf.d/#{new_resource.site_name}.conf" do
    source 'site.conf.erb'
    cookbook 'webserver'
    variables(site: new_resource.site_name, port: new_resource.port)
    notifies :reload, 'service[nginx]', :delayed
  end

  file "/var/www/#{new_resource.site_name}/index.html" do
    content "<h1>#{new_resource.message}</h1>\n<p>Site #{new_resource.site_name} on #{node['hostname']}, #{node['platform']} #{node['platform_version']}.</p>\n"
  end
end

action :delete do
  file "/etc/nginx/conf.d/#{new_resource.site_name}.conf" do
    action :delete
    notifies :reload, 'service[nginx]', :delayed
  end

  directory "/var/www/#{new_resource.site_name}" do
    recursive true
    action :delete
  end
end
Properties have types and defaults (property :port, Integer, default: 80), and name_property: true means "the title of the resource is the value". Inside an action you use the ordinary resources, and you read the values with new_resource. The whole bundle then shows up as ONE resource in the output, with its inner resources nested below it:
  * webserver_site[shop] action create
    * directory[/var/www/shop] action create
    * template[/etc/nginx/conf.d/shop.conf] action create
    * file[/var/www/shop/index.html] action create
Properties are checked before anything runs:
$ (a recipe with: webserver_site 'broken')
  * webserver_site[broken] action create
    Chef::Exceptions::ValidationFailed
    message is a required property
$ (a recipe with: port 'eighty')
Recipe Compile Error in /root/.cinc/local-mode-cache/cache/cookbooks/webserver/recipes/bad_type.rb
Chef::Exceptions::ValidationFailed
Property port must be one of: Integer!  You passed "eighty".
A wrong type is caught while the recipe is READ (a compile error: nothing was applied), a missing required property when the resource is reached.
And a trap in the delete action. I ran it from a recipe that only said webserver_site 'blog' do ... action :delete end, in a run that did not include the default recipe. The files were deleted, and then the run FAILED:
  * webserver_site[blog] action delete
    * file[/etc/nginx/conf.d/blog.conf] action delete
      - delete file /etc/nginx/conf.d/blog.conf
    * directory[/var/www/blog] action delete
      - delete existing directory /var/www/blog
    Chef::Exceptions::MultipleFailures
    Multiple failures occurred:
    * Chef::Exceptions::ResourceNotFound occurred in Cinc Client run: resource file[/etc/nginx/conf.d/blog.conf] is configured to notify resource service[nginx] with action reload, but service[nginx] cannot be ...
    * NoMethodError occurred in delayed notification: undefined method 'run_action' for an instance of String
(exit 1.) The notification names service[nginx], and that resource only exists when the run list also contains the recipe that declares it. A notification target must be in the same run. A custom resource that notifies something outside itself is coupled to whoever declares it: either declare it inside the custom resource, or make sure every run list includes it.

Attributes are the node's data. They are set at several levels, and when two levels set the same key a fixed order decides. Chef can print the whole stack for one key, node.debug_value, which is the quickest way to answer "where did this value come from?":
$ cat recipes/debug.rb
puts "  node['webserver']['port'] = #{node['webserver']['port']}"
node.debug_value('webserver', 'port').each do |level, value|
  puts "    #{level.ljust(18)} #{value.inspect}" unless value == :not_present
end
I set port in the cookbook's attributes file (80), in a role (8081), in an environment (production: 8082), as JSON given to the run (8083), and in a second environment (staging: default 8084 and an override of 9000), then ran the debug recipe with different combinations:
$ cinc-client -z -N web9.lab -o 'recipe[webserver::debug]'
  node['webserver']['port'] = 80
    default            80
$ cinc-client -z -N web9.lab -o 'role[web],recipe[webserver::debug]'
  node['webserver']['port'] = 8081
    default            80
    role_default       8081
$ cinc-client -z -N web9.lab -E production -o 'role[web],recipe[webserver::debug]'
  node['webserver']['port'] = 8081
    default            80
    env_default        8082
    role_default       8081
$ cinc-client -z -N web9.lab -E production -o 'recipe[webserver::debug]'
  node['webserver']['port'] = 8082
    default            80
    env_default        8082
$ cinc-client -z -N web9.lab -E production -j attrs.json -o 'role[web],recipe[webserver::debug]'
  node['webserver']['port'] = 8083
    default            80
    env_default        8082
    role_default       8081
    normal             8083
$ cinc-client -z -N web9.lab -E staging -j attrs.json -o 'role[web],recipe[webserver::debug]'
  node['webserver']['port'] = 9000
    default            80
    env_default        8084
    role_default       8081
    normal             8083
    env_override       9000
Read the lists from top to bottom as "weakest to strongest": the cookbook default is beaten by the environment default, which is beaten by the role default (8081 won over 8082), which is beaten by "normal" (the JSON of the run, 8083), which is beaten by the environment override (9000). There are more levels (the node's own default/override, the automatic attributes from Ohai, and force_override); only the ones above were tested. The practical rules: put the safe default in the cookbook's attributes file; let roles and environments change it; do not use override unless you mean "nothing may change this".
(One thing to know about that experiment: the runs with role[web] in the run list also CONVERGED the role's recipe, so each of them really changed the machine's port. The last one left nginx listening on 9000, which the audit section below stumbled over.)

  role          a named run list with attributes: "a web server". A node's run list can contain role[web]
  environment   a named set of attributes and cookbook version constraints: production, staging
  data bag      a named collection of JSON items, for data that belongs to no cookbook: users, certificates, secrets
In a repository, they are JSON files in roles/, environments/ and data_bags/NAME/. The role of this page:
$ cat roles/web.json
{
  "name": "web",
  "description": "A web server",
  "json_class": "Chef::Role",
  "chef_type": "role",
  "default_attributes": { "webserver": { "port": 8081 } },
  "run_list": [ "recipe[webserver]" ]
}
Data bags, listed and read with knife in local mode (-z, which uses the files of the repository instead of a server):
$ knife data bag list -z
users
$ knife data bag show users -z
alice
bob
$ knife data bag show users alice -z
groups: web
id:     alice
shell:  /bin/bash
Search works in local mode too, and a recipe can use it to find items:
$ cat recipes/users.rb
search(:users, 'groups:web').each do |u|
  user u['id'] do
    shell u['shell']
    manage_home true
    comment 'Created by Chef from a data bag'
  end
end

secret = data_bag_item('secrets', 'db', IO.read('/root/repo/secret.key').strip)
file '/etc/db-password' do
  content "#{secret['password']}\n"
  mode '0600'
  sensitive true
end
$ getent passwd alice bob
alice:x:1001:1001:Created by Chef from a data bag:/home/alice:/bin/bash
Only alice was created: the query groups:web matched her item and not bob's (groups: db). The second half uses an ENCRYPTED data bag. The item was written with a secret key:
$ knife data bag from file secrets /tmp/db.json --secret-file secret.key -z
Updated data_bag_item[secrets::db]
$ cat data_bags/secrets/db.json
{
  "id": "db",
  "password": {
    "encrypted_data": "ZqN+drGh6bNRjdMEL6lOfOkchlwDwMV+JDb1EGZt9+x1DQ==\n",
    "iv": "k8cZSv6SF3+LaefY\n",
    "auth_tag": "lL1Rq0MsEREW0hgxm1AREA==\n",
    "version": 3,
    "cipher": "aes-256-gcm"
  }
}
Only the VALUES are encrypted: the id and the key names stay readable, so you can still see which items exist. Without the key knife prints the ciphertext and a warning ("Encrypted data bag detected, but no secret provided for decoding"); with it:
$ knife data bag show secrets db --secret-file secret.key -z
Encrypted data bag detected, decrypting with provided secret.
id:       db
password: S3cret-Pass-123
and the recipe wrote /etc/db-password with mode 600 and "suppressed sensitive resource" in the output instead of a diff, because the file resource says sensitive true. The shared secret key itself must be delivered to the nodes and kept out of the repository; how to do that was not run in the lab (a secrets manager or chef-vault, which the workstation includes, is the usual answer).

The client can run with an in-memory server of its own, from the files of a repository: cinc-client -z (--local-mode). That is what every "-z" command above did, and it is the easiest way to test a whole run list with roles, environments, data bags and search, with nothing installed on a server:
$ cinc-client -z -o "recipe[webserver]"
Infra Phase complete, 16/20 resources updated in 09 seconds
(-o overrides the run list for this one run and, as a safety, does not save the node: "Skipping final node save because override_runlist was given".) Run again, and it is quiet: "Infra Phase complete, 0/15 resources updated in 02 seconds".
For several machines you want one server that they all ask. The workstation contains one for development, cinc-zero (the Cinc build of chef-zero):
$ cinc-zero --host 0.0.0.0 --port 8889
>> Starting Cinc Zero (v15.1.11)...
>> WEBrick (v1.9.2) on Rack (v3.2.7) is listening at http://0.0.0.0:8889
Point knife at it. The configuration file is ~/.cinc/config.rb: the Cinc tools look in .cinc, not .chef. I first put it in ~/.chef/ and every knife command answered "WARNING: No knife configuration file found" and then complained about a private key at /etc/cinc/client.pem. (Chef's own tools use .chef; not run.)
$ cat ~/.cinc/config.rb
node_name        'admin'
client_key       '/root/.cinc/admin.pem'
chef_server_url  'http://chef:8889'
cookbook_path    ['/root/repo/cookbooks']
Then upload what the nodes need:
$ knife cookbook upload webserver
Uploading webserver    [0.1.0]
Uploaded 1 cookbook.
$ knife role from file roles/web.json
Updated Role web
$ knife environment from file environments/production.json
Updated Environment production
$ knife data bag create users
Created data_bag[users]
$ knife data bag from file users data_bags/users/alice.json data_bags/users/bob.json
Updated data_bag_item[users::alice]
Updated data_bag_item[users::bob]
$ knife cookbook list; knife role list; knife environment list; knife data bag list
webserver: 0.1.0
web
_default
production
users
A new machine gets its identity in one of two ways: it uses a "validator" key to register itself, or an administrator creates a client for it and hands over its key. With a client.rb that names the server and the node, and a first-boot file with the run list:
$ cat /etc/cinc/client.rb
chef_server_url "http://chef:8889"
node_name       "node1.lab"
environment     "production"
$ cat /etc/cinc/first-boot.json
{ "run_list": ["role[web]"] }
the first run with no key anywhere fails, and the message is clear:
$ cinc-client -j /etc/cinc/first-boot.json
Creating a new client identity for node1.lab using the validator key.
WARN: Failed to read the private key /etc/cinc/validation.pem: #<Errno::ENOENT: No such file or directory @ rb_sysopen - /etc/cinc/validation.pem>
Private Key Not Found:
Your private key could not be loaded. If the key file exists, ensure that it is
readable by Cinc Client.
So create the client on the workstation, and copy its key to the node (knife client create wants to open an editor unless you pass -d, --disable-editing):
$ knife client create node1.lab -d -f /tmp/node1.pem
Created client[node1.lab]
$ knife client list
chef-validator
chef-webui
node1.lab
and copy /tmp/node1.pem to /etc/cinc/client.pem on node1, mode 600. Now the node converges from the server:
$ cinc-client -j /etc/cinc/first-boot.json
Cinc Client, version 19.3.14
Infra Phase starting
Resolving cookbooks for run list: ["webserver"]
Synchronizing cookbooks:
  - webserver (0.1.0)
Converging 8 resources
Recipe: webserver::default
  * apt_update[refresh] action periodic (up to date)
  * apt_package[nginx] action install
    - install version 1.24.0-2ubuntu7.18 of package nginx
  ...
  * service[nginx] action reload
    - reload service service[nginx]
Infra Phase complete, 13/16 resources updated in 07 seconds
The server now knows the node, and you can ask it about it:
$ knife node list
node1.lab
$ knife node show node1.lab
Node Name:   node1.lab
Environment: production
FQDN:        node1.lab
IP:          172.21.0.2
Run List:    role[web]
Roles:       web
Recipes:     webserver, webserver::default
Platform:    ubuntu 24.04
Tags:
$ knife search node "role:web" -a platform -a ipaddress -a webserver.port
1 items found

node1.lab:
  ipaddress:      172.21.0.2
  platform:       ubuntu
  webserver.port: 8081
$ knife search node "platform:ubuntu AND chef_environment:production" -i
1 items found

node1.lab
Note the port: 8081, the ROLE's default, won over the production environment's 8082, as in the attribute test above. Now change the role on the server (8081 to 8085) and let the node pick it up on its next run:
$ knife role from file roles/web.json
Updated Role web
$ cinc-client -j /etc/cinc/first-boot.json          (on node1)
  * webserver_site[shop] action create
    * template[/etc/nginx/conf.d/shop.conf] action create
      - update content in file /etc/nginx/conf.d/shop.conf from f31983 to 30fa93
      --- /etc/nginx/conf.d/shop.conf	2026-10-09 13:40:37.658355711 +0000
      +++ /etc/nginx/conf.d/.chef-shop20261009-3863-jgopvo.conf	2026-10-09 13:41:03.746779855 +0000
      @@ -1,6 +1,6 @@
       # Managed by Chef (cookbook webserver)
       server {
      -    listen 8081;
      +    listen 8085;
That is the loop: change the data on the server, the client pulls and converges. (Nothing runs the client by itself in these containers; in real life it runs as a service or from cron, every 30 minutes by default, which was not run in the lab.) Right after that run, ss showed both 8081 and 8085 listening: the old workers were still shutting down (a graceful reload); a moment later only 8085 was left.
The warning I owe you about this server: cinc-zero checks nobody. I sent knife requests with the admin key, with a key the server had never seen, with a key and the name of an existing client, and with the name of a client that does not exist:
$ knife node list --key /tmp/random.pem          (a key the server never saw)
node1.lab
node2.lab
$ knife node list -u nosuchclient --key /tmp/random.pem
node1.lab
node2.lab
All four answered with the list of nodes. It is a development server, and nothing on this page about keys and clients describes a real, authenticating server. A real Chef or Cinc Server has organisations, users, signed requests and TLS. I tried to run one and could not (next section).

Roles and environments let different machines see different cookbook versions and attributes at different times, which makes "what exactly runs on this node?" hard to answer. A Policyfile pins it: one file lists the run list and the cookbooks, a lock file records the exact versions, and the whole is pushed to the server under a name and a policy group. Nodes then say which policy and group they follow, with no run list of their own.
$ cat Policyfile.rb
name 'webserver'
default_source :supermarket
run_list 'webserver::default'
cookbook 'webserver', path: 'cookbooks/webserver'
$ cinc install Policyfile.rb
Building policy webserver
Expanded run list: recipe[webserver::default]
Caching Cookbooks...
Installing webserver >= 0.0.0 from path

Lockfile written to /root/repo/Policyfile.lock.json
Policy revision id: 2ce67cbb834ce5ac342d7a37d115b0f5a82158bbc742c8eb5e2d52b9a4a9b5b4
The lock file records the cookbook with a content identifier ("864b8f32df351c1557711cb3120cf5eafa54cd5f") so that "version 0.1.0" can no longer mean two different things. Push it to a policy group, here "production":
$ cinc push production Policyfile.rb
Uploading policy webserver (2ce67cbb83) to policy group production
Uploaded webserver 0.1.0 (864b8f32)
$ cinc show-policy webserver
webserver
=========

* production:  2ce67cbb83
A third machine (node2.lab) follows it, with a client.rb that names the policy and the group and nothing else:
$ cat /etc/cinc/client.rb
chef_server_url "http://chef:8889"
node_name       "node2.lab"
policy_name     "webserver"
policy_group    "production"
$ cinc-client
Cinc Client, version 19.3.14
Infra Phase starting
Using Policyfile 'webserver' at revision '2ce67cbb834ce5ac342d7a37d115b0f5a82158bbc742c8eb5e2d52b9a4a9b5b4'
Resolving cookbooks for run list: ["webserver::default@0.1.0 (864b8f3)"]
Synchronizing cookbooks:
  - webserver (0.1.0)
Infra Phase complete, 13/16 resources updated in 08 seconds
The node ran exactly revision 2ce67cbb83. (Policyfile dependencies from the Supermarket, policy inheritance and promoting a policy from one group to the next were not run in the lab.)

Cookstyle is a RuboCop with Chef's rules. It passed the cookbook of this page without an offense ("4 files inspected, no offenses detected"), and it finds plenty in a careless recipe:
$ cat recipes/default.rb
package "curl" do
  action :install
end

execute "restart-it" do
  command "service nginx restart"
end

service "nginx" do
  action [ :enable, :start ]
end

file "/tmp/x" do
  mode 0644
  content "hello"
end
$ cookstyle .          (each message line continues with a link to the rule's documentation, cut here)
recipes/default.rb:1:9: C: [Correctable] Style/StringLiterals: Prefer single-quoted strings when you don't need string interpolation or special symbols.
recipes/default.rb:6:11: R: Chef/Correctness/ServiceResource: Use the service resource to manage services instead of shelling out to an init script, service, systemctl, or a similar tool
recipes/default.rb:14:8: R: [Correctable] Chef/Style/FileMode: Use strings to represent file modes to avoid confusion between octal and base 10 integer formats
...
2 files inspected, 8 offenses detected, 7 offenses autocorrectable
$ cookstyle -a .
2 files inspected, 8 offenses detected, 7 offenses corrected
The autocorrect fixed seven. The eighth, shelling out to "service nginx restart" in an execute, it cannot fix: it asks you to use the service resource, which is the right answer. Notice what it did to the mode:
file '/tmp/x' do
  mode '644'
  content 'hello'
end
mode 0644 became '644' (a string), not '0644'. Chef reads a string mode as octal, so the file mode is the same; I did not test the result beyond the lint.
Cinc Auditor (InSpec) checks the state of a machine with tests, and is how you verify a converge. A small test of the web server:
$ cat audit_test.rb
describe port(80) do
  it { should be_listening }
end

describe port(8080) do
  it { should be_listening }
end

describe file('/etc/nginx/conf.d/shop.conf') do
  it { should exist }
  its('mode') { should cmp '0644' }
  its('content') { should match(/server_name shop\.example\.org;/) }
end

describe command('curl -s -H "Host: shop.example.org" localhost/') do
  its('stdout') { should match(/Welcome to the shop/) }
end
I wrote the 8080 test to fail. The first run failed twice more than I expected:
$ cinc-auditor exec audit_test.rb
  Port 80
     ×  is expected to be listening
  Port 8080
     ×  is expected to be listening
  File /etc/nginx/conf.d/shop.conf
     ✔  is expected to exist
     ✔  mode is expected to cmp == "0644"
     ✔  content is expected to match /server_name shop\.example\.org;/
  Command: `curl -s -H "Host: shop.example.org" localhost/`
     ×  stdout is expected to match /Welcome to the shop/
Test Summary: 3 successful, 3 failures, 0 skipped
Port 80 closed and no page: the audit was right and I had forgotten what I had done to the machine. The attribute experiments (see above) had applied the staging environment's override, and nginx had been listening on 9000 ever since. Converging the plain recipe again put it back:
$ ss -ltn | grep -E "0.0.0.0:(80|8080|9000) "
LISTEN 0      511          0.0.0.0:9000       0.0.0.0:*
$ cinc-client -z -o "recipe[webserver]"
      -    listen 9000;
      +    listen 80;
    - reload service service[nginx]
Infra Phase complete, 5/16 resources updated in 03 seconds
$ cinc-auditor exec audit_test.rb
  Port 80
     ✔  is expected to be listening
  Port 8080
     ×  is expected to be listening
  ...
Test Summary: 5 successful, 1 failure, 0 skipped
(exit code 100 while any test fails.) That is the use of an audit: not "did the converge print green", but "is the machine in the state we want". I passed --chef-license accept-silent to cinc-auditor to avoid any prompt; whether it asks without the flag was not tested.

Every one of these ran in the lab, from a small recipe. The first line of each is what the client printed.
  SyntaxError: syntax.rb:3: syntax errors found       an unclosed block (a do without an end). The next lines of the message point at it:
      | ^ expected a block beginning with `do` to end with `end`
  NoMethodError: undefined method 'pakage' for an instance of Chef::Recipe       a typo in a resource name
  NoMethodError: undefined method `contnet' for Chef::Resource::File       a typo in a property name
  Chef::Exceptions::ValidationFailed: Option action must be equal to one of: :nothing, :install, :upgrade, :remove, :purge, :reconfig, :lock, :unlock!  You passed :instal.       a typo in an action
  Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'       an execute that failed. Chef STOPS at the first failed resource: the file resource after it in my test was not created (ls /tmp/after-boom: No such file or directory)
  NoMethodError: undefined method 'preferred_filename_on_disk_location' for nil       a template whose source file does not exist in the cookbook (the message is confusing: the cause is the missing template file)
  Chef::Exceptions::ResourceNotFound: resource file[/tmp/n] is configured to notify resource service[nosuch] with action restart, but service[nosuch] cannot be found in the resource collection       a notification to a resource that is not in the run
  NoMethodError: undefined method '[]' for nil       reading node['nosuch']['deeper'] when node['nosuch'] does not exist: the first lookup returned nil. Check each level, or use node.dig('nosuch', 'deeper')
  NoMethodError: undefined method '[]=' for an instance of Integer       node.default['a'] = 1 and then node.default['a']['b'] = 2: a value that is a number cannot have sub-keys
  message is a required property / Property port must be one of: Integer!  You passed "eighty".       custom resource properties (shown above)
  Private Key Not Found (Creating a new client identity for node1.lab using the validator key)       the node has no key and no validator: create the client with knife and copy the key
  ERROR: Your private key could not be loaded from /etc/cinc/client.pem       knife has no usable configuration: for the Cinc tools it belongs in ~/.cinc/config.rb
  Please set EDITOR environment variable       knife client create (and others) open an editor: add -d (--disable-editing)
node.dig was not run in the lab; it is the standard Ruby/Chef way to read nested keys, and I mention it for the fix of the nil error above.

  - The real Chef Infra Server (here the Cinc Server, package cinc-server-core 15.10.125, which installed). Its reconfigure step needs systemd: the first try died on "Errno::ENOENT: No such file or directory - systemctl". The second, with a stub systemctl that did nothing and a hand-started runit supervisor, died on '/usr/bin/systemctl show' not reporting status for private_chef-runsvdir-start.service!. The third, with a stub that answered the status question, died on the same error. I stopped there and removed the container. So everything in the client/server section was done against cinc-zero, which authenticates nobody. Organisations, users, signed requests, TLS and the server's own services were not run
  - Anything under systemd: the systemd service provider (this lab used the init-script one), service reloads on a real host
  - Chef's own packages, Chef Workstation, Chef's license acceptance, and any version other than Cinc Client 19.3.14 and Cinc Workstation 26.3.0. Chef's own download service listed a different version (18.11.11) for its stable client when I looked, and I did not look into why
  - Test Kitchen (kitchen.yml was generated but needs a VM or container driver), ChefSpec, Berkshelf and the Supermarket, Policyfile dependencies from the Supermarket, chef-vault, knife bootstrap and knife ssh
  - The client as a daemon or from cron, handlers and reporting, Automate, and anything about scale
  - Windows, other Linux families, macOS
  - cinc-client -W (why-run) with a server and with cookbooks (only with cinc-apply and with -z), the node.dig fix, and string versus integer file modes beyond lint
  - Custom resource features beyond properties and two actions: converge_if_changed, load_current_value, resource inheritance, partial templates, library helpers

Preview a recipe file:                           cinc-apply web.rb -W
Run it:                                          cinc-apply web.rb
A cookbook without any server:                   cinc-client -z -o 'recipe[webserver]'
The same, as a preview:                          cinc-client -z -o 'recipe[webserver]' -W
Where does this value come from?                 node.debug_value('webserver', 'port')   (inside a recipe), then print it
A role and an environment, no server:            cinc-client -z -N NAME -E production -o 'role[web]'
Facts of this machine:                           ohai platform ipaddress memory/total
Lint, and fix what can be fixed:                 cookstyle .        cookstyle -a .
Test the machine, not the converge:              cinc-auditor exec test.rb
Make a cookbook, a recipe, a resource:           cinc generate cookbook NAME     cinc generate recipe NAME    cinc generate resource NAME
Push a cookbook, role, environment, data bag:    knife cookbook upload NAME     knife role from file F     knife environment from file F     knife data bag from file BAG F
Make a client key for a node:                    knife client create NODE -d -f NODE.pem
What does the server know about the node?        knife node show NODE      knife search node 'role:web' -a platform
Pin it all with a Policyfile:                    cinc install Policyfile.rb   then   cinc push GROUP Policyfile.rb
Run what the server says for this node:         cinc-client

Symptom                                          Usually
Works on the second run only                     plain Ruby (an if, a File.exist?) that ran at compile time, before the resource it depends on. Use a guard or a ruby_block
The service did not pick up the change           init-script systems: :reload without supports reload: true does nothing, silently. Verify the effect, not the log line. Or use :restart
The site shows the old page right after the run  the reload is asynchronous: wait a moment or retry
A recipe notifies a resource and fails           the target resource is not in this run (not in the run list, or declared in another recipe): notifications need it in the same run
NoMethodError: undefined method 'x' for ...::RubyBlock   a def in the recipe is not visible inside a resource block: use a lambda or a local variable
A notification never fires                       the notifying resource never "changes": a log resource does not count
Every run says execute was updated               an execute without creates, not_if or only_if
Why is the port 8081 and not 80 or 8082?        run node.debug_value for the key: the role default beats the environment default and the cookbook default; JSON given to the run beats those; an environment override beats everything shown on this page
cinc-apply "No file exists at .../--no-color"    the recipe file must come first, then the options
knife says no configuration file found           the Cinc tools read ~/.cinc/config.rb (not ~/.chef/)
knife client create asks for an editor           add -d
A new node cannot register                       no validator key and no client key: create the client with knife and copy its key to /etc/cinc/client.pem
The "server" accepts everything                  that is cinc-zero, a development server: it checks no keys
A test fails after experiments                   the machine is not in the state you think: converge again, then audit
A stopped service stays "defunct" in a container   no init process in the container: docker run --init

Think in two phases. Anything that depends on the state of the machine goes in a guard, a lazy value or a ruby_block, never in plain Ruby at the top of a recipe.
Keep data in attributes and the logic in recipes and resources; keep the cookbook's defaults sensible and let roles, environments or a Policyfile change them.
Put repeatable work in custom resources, with typed properties and a name.
Verify the effect, not the log: count the processes, ask the site, run a Cinc Auditor test. A converge that prints "up to date" only proves that Chef thinks it is done.
Guard every execute. Use real resources whenever one exists.
Use node.debug_value when a value surprises you; it is faster than reading every file.
Lint with Cookstyle in your pipeline, and test with Cinc Auditor, and with Test Kitchen where you can (not run here).
Pin what a node runs: a Policyfile, not a pile of roles and environments.
Use a real, authenticating server in any shared setup. cinc-zero and local mode are for development and for tests.
Keep secrets in encrypted data bags or a secrets manager, and the key out of the repository. Mark resources that carry secrets sensitive.
Only manage machines you are authorized to manage: whoever controls the cookbooks controls the machines.