Deprecated: Creation of dynamic property c2c_AddAdminCSS::$admin_options_name is deprecated in /var/www/html/wp-content/plugins/add-admin-css/c2c-plugin.php on line 106 Deprecated: Creation of dynamic property c2c_AddAdminCSS::$config is deprecated in /var/www/html/wp-content/plugins/add-admin-css/c2c-plugin.php on line 106 Deprecated: Creation of dynamic property c2c_AddAdminCSS::$disable_contextual_help is deprecated in /var/www/html/wp-content/plugins/add-admin-css/c2c-plugin.php on line 106 Deprecated: Creation of dynamic property c2c_AddAdminCSS::$disable_update_check is deprecated in /var/www/html/wp-content/plugins/add-admin-css/c2c-plugin.php on line 106 Deprecated: Creation of dynamic property c2c_AddAdminCSS::$hook_prefix is deprecated in /var/www/html/wp-content/plugins/add-admin-css/c2c-plugin.php on line 106 Deprecated: Creation of dynamic property c2c_AddAdminCSS::$form_name is deprecated in /var/www/html/wp-content/plugins/add-admin-css/c2c-plugin.php on line 106 Deprecated: Creation of dynamic property c2c_AddAdminCSS::$menu_name is deprecated in /var/www/html/wp-content/plugins/add-admin-css/c2c-plugin.php on line 106 Deprecated: Creation of dynamic property c2c_AddAdminCSS::$name is deprecated in /var/www/html/wp-content/plugins/add-admin-css/c2c-plugin.php on line 106 Deprecated: Creation of dynamic property c2c_AddAdminCSS::$nonce_field is deprecated in /var/www/html/wp-content/plugins/add-admin-css/c2c-plugin.php on line 106 Deprecated: Creation of dynamic property c2c_AddAdminCSS::$settings_page is deprecated in /var/www/html/wp-content/plugins/add-admin-css/c2c-plugin.php on line 106 Deprecated: Creation of dynamic property c2c_AddAdminCSS::$show_admin is deprecated in /var/www/html/wp-content/plugins/add-admin-css/c2c-plugin.php on line 106 Deprecated: Creation of dynamic property c2c_AddAdminCSS::$textdomain is deprecated in /var/www/html/wp-content/plugins/add-admin-css/c2c-plugin.php on line 106 Deprecated: Creation of dynamic property c2c_AddAdminCSS::$textdomain_subdir is deprecated in /var/www/html/wp-content/plugins/add-admin-css/c2c-plugin.php on line 106 Deprecated: Creation of dynamic property c2c_AddAdminCSS::$author_prefix is deprecated in /var/www/html/wp-content/plugins/add-admin-css/c2c-plugin.php on line 109 Deprecated: Creation of dynamic property c2c_AddAdminCSS::$id_base is deprecated in /var/www/html/wp-content/plugins/add-admin-css/c2c-plugin.php on line 110 Deprecated: Creation of dynamic property c2c_AddAdminCSS::$options_page is deprecated in /var/www/html/wp-content/plugins/add-admin-css/c2c-plugin.php on line 111 Deprecated: Creation of dynamic property c2c_AddAdminCSS::$plugin_basename is deprecated in /var/www/html/wp-content/plugins/add-admin-css/c2c-plugin.php on line 112 Deprecated: Creation of dynamic property c2c_AddAdminCSS::$plugin_file is deprecated in /var/www/html/wp-content/plugins/add-admin-css/c2c-plugin.php on line 113 Deprecated: Creation of dynamic property c2c_AddAdminCSS::$plugin_path is deprecated in /var/www/html/wp-content/plugins/add-admin-css/c2c-plugin.php on line 114 Deprecated: Creation of dynamic property c2c_AddAdminCSS::$u_id_base is deprecated in /var/www/html/wp-content/plugins/add-admin-css/c2c-plugin.php on line 115 Deprecated: Creation of dynamic property c2c_AddAdminCSS::$version is deprecated in /var/www/html/wp-content/plugins/add-admin-css/c2c-plugin.php on line 116 Warning: Cannot modify header information - headers already sent by (output started at /var/www/html/wp-content/plugins/add-admin-css/c2c-plugin.php:106) in /var/www/html/wp-includes/rest-api/class-wp-rest-server.php on line 1768 {"id":217,"date":"2022-08-12T20:00:17","date_gmt":"2022-08-12T13:00:17","guid":{"rendered":"https:\/\/erwin.co\/?p=217"},"modified":"2022-08-12T20:00:17","modified_gmt":"2022-08-12T13:00:17","slug":"how-do-variables-really-work-in-dockerfiles","status":"publish","type":"post","link":"https:\/\/erwin.co\/how-do-variables-really-work-in-dockerfiles\/","title":{"rendered":"How do variables really work in Dockerfiles?"},"content":{"rendered":"\n

Whether you're naming a Dockerfile ARG or ENV variable or a regular shell script variable, inside your Dockerfile they're all referenced as simply $var<\/strong>. For Docker Compose commands there's a special $$var syntax for variables<\/a> that you don't want Docker Compose to interpolate.<\/p>\n\n\n\n

Every docker RUN<\/code> command is a completely separate process\/environment, so if you're using a regular shell script variable, setting and getting the variable must all be done within one RUN<\/code> command.<\/p>\n\n\n\n

If you need to share a shell script variable $var<\/code> across several RUN<\/code> commands, it's definitely preferable to try and 1) refactor it into a single RUN<\/code> command or 2) set the value in a build-arg<\/code> and write a wrapper script. If neither of those work, then you should just read and write your variable value to\/from files. For example:<\/p>\n\n\n

\nRUN echo 1 > \/tmp\/__var_1\nRUN echo `cat \/tmp\/__var_1`\nRUN rm -f \/tmp\/__var_1\n<\/pre><\/div>\n\n\n
If your RUN<\/code> command works in your shell but not via your Dockerfile RUN<\/code>, it's likely a quoting issue.<\/p>\n\n\n\n
  1. Docker RUN<\/code> commands are hard coded<\/a> to use \/bin\/sh -c<\/code>. On many systems sh<\/code> will be dash<\/code> or ash<\/code> or another very minimal shell with slightly different rules than the shell you typically use.<\/li>
  2. Use a RUN echo ...<\/code> command to make sure that any variables your command depends on are in fact set to what you think they're set to.<\/li>
  3. Try to make the command you're having trouble with the first command in your file so that you can keep running your docker build<\/code> with --no-cache<\/code> and still minimize your wait time.<\/li>
  4. When I was troubleshooting what also ended up being a quoting problem, I should have written my simplified test version like so:<\/li><\/ol>\n\n\n
    \nRUN SHELL_PATH=$(head -n 1 \/etc\/shells) &&\\\n    useradd --shell $SHELL_PATH --uid 1000 foo\n<\/pre><\/div>\n\n\n
    Even if you use zsh<\/code> as your normal shell, I would recommend using bash to test your RUN<\/code> commands like:<\/p>\n\n\n
    \n\/bin\/sh -c 'SHELL_PATH=$(head -n 1 \/etc\/shells) &&\\\n    useradd --shell $SHELL_PATH --uid 1000 foo'\n<\/pre><\/div>\n\n\n
    1. Dockerfile ARG<\/code> values will overwrite any shell script variables that you set\u00e2\u20ac\u00a6 For example, say we have this Dockerfile<\/code><\/li><\/ol>\n\n\n
      \nFROM alpine:latest\n\nARG ArgFoo\nENV EnvFoo="Must be Set"\n\nRUN echo "Value of ArgFoo is $ArgFoo"\nRUN echo "Value of EnvFoo is $EnvFoo"\nRUN ShFoo="awesome" && echo "Value of ShFoo is $ShFoo"\nRUN echo "Our \\$ShFoo is Gone Again: $ShFoo"\n<\/pre><\/div>\n\n\n
      When we run:<\/p>\n\n\n
      \n$ docker build . --no-cache\n\nStep 4\/7 : RUN echo "Value of ArgFoo is $ArgFoo"\nValue of ArgFoo is\n\nStep 5\/7 : RUN echo "Value of EnvFoo is $EnvFoo"\nValue of EnvFoo is Must be Set\n\nStep 6\/7 : RUN ShFoo="awesome" && echo "Value of ShFoo is $ShFoo"\nValue of ShFoo is awesome\n\nStep 7\/7 : RUN echo "Our \\$ShFoo is Gone Again: $ShFoo"\nOur $ShFoo is Gone Again:\n<\/pre><\/div>\n\n\n
      If we ran it again, but this time with a --build-arg<\/code> setting $ShFoo<\/code> surprisingly, it's still difficult to get ourselves into trouble. First, update the Dockerfile to try and cause some trouble with $ShFoo<\/code><\/p>\n\n\n
      \nFROM alpine:latest\n\nARG ShFoo\nRUN echo "$ShFoo" && ShFoo="awesome" && echo "Value of ShFoo is $ShFoo"\n\n<\/pre><\/div>\n\n
      \n$ docker build . --build-arg ShFoo="Difficult to cause trouble" --no-cache\n\nStep 3\/3 : RUN echo "$ShFoo" && ShFoo="awesome" && echo "Value of ShFoo is $ShFoo"\nDifficult to cause trouble\nValue of ShFoo is awesome\n<\/pre><\/div>\n\n\n
      So Docker does a surprisingly good job of allowing you to reference any variable as just $var<\/code> and everything just working<\/em>.<\/p>\n\n\n\n
      The main difference when writing RUN<\/code> commands really has more to do with \/bin\/sh -c<\/code> than it does with Docker.<\/p>\n\n\n\n
      For example, I was working on a Dockerfile<\/code> that would automatically set the permissions of the running Docker container to match the current user.  Ultimately the command that worked was like this:<\/p>\n\n\n
      \nRUN shell=$(grep -E -m 1 \\.\\*\\\\b$USER_SHELL\\\\b \/etc\/shells) && \\\n    echo "DUMP: $shell $USER_ID:$GROUP_ID $USER_NAME:$GROUP_NAME" && \\\n    groupadd --gid $GROUP_ID $GROUP_NAME && \\  \n    useradd --shell $shell --uid $USER_ID --gid $GROUP_ID $USER_NAME\n<\/pre><\/div>\n\n\n
      The subshell portion is:<\/p>\n\n\n
      \ngrep -E -m 1 \\.\\*\\\\b$USER_SHELL\\\\b \/etc\/shells\n<\/pre><\/div>\n\n\n
      Yet when executing in a normal shell I only need to use:<\/p>\n\n\n
      \ngrep -E -m 1 '.*\\bzsh\\b' \/etc\/shells\n<\/pre><\/div>\n\n\n
      When you have a fairly complex shell command inside of a docker RUN<\/code> the format above of assigning a shell variable, then echoing everything so that you can be sure you really have what you think you have, is probably a good way to go if you start running into anything unexpected.<\/p>\n","protected":false},"excerpt":{"rendered":"
      Whether you’re naming a Dockerfile ARG or ENV variable or a regular shell script variable, inside your Dockerfile they’re all referenced as simply $var. For Docker Compose commands there’s a special $$var syntax for variables that you don’t want Docker Compose to interpolate. Every docker RUN command is a completely separate process\/environment, so if you’re […]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"closed","ping_status":"","sticky":false,"template":"","format":"standard","meta":[],"categories":[1],"tags":[],"_links":{"self":[{"href":"https:\/\/erwin.co\/wp-json\/wp\/v2\/posts\/217"}],"collection":[{"href":"https:\/\/erwin.co\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/erwin.co\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/erwin.co\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/erwin.co\/wp-json\/wp\/v2\/comments?post=217"}],"version-history":[{"count":1,"href":"https:\/\/erwin.co\/wp-json\/wp\/v2\/posts\/217\/revisions"}],"predecessor-version":[{"id":218,"href":"https:\/\/erwin.co\/wp-json\/wp\/v2\/posts\/217\/revisions\/218"}],"wp:attachment":[{"href":"https:\/\/erwin.co\/wp-json\/wp\/v2\/media?parent=217"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/erwin.co\/wp-json\/wp\/v2\/categories?post=217"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/erwin.co\/wp-json\/wp\/v2\/tags?post=217"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}