Skip to Content

GitHub API limit pushs me to learn how to use docker multi-stage build to hide secret

1 cause

GitHub API limit (when installing R packages from GitHub) on Travis CI has bothered me for a while, using personal token seems to be a nice solution.

But for Docker, that’s a problem.

2 strugle

At first, I spend some time to realize that it’s impossible to ask Dockerfile to use environment variables on host machine 1.

3 seemingly promissing dead end

Then I found a seemingly nice approch:

  • add ARG GITHUB_PAT to Dockerfile,
  • use docker build --build-arg GITHUB_PAT=${GITHUB_PAT} ...

But the official documentation warns me:

It is not recommended to use build-time variables for passing secrets like github keys, user credentials etc. Build-time variable values are visible to any user of the image with the docker history command.

4 success, yet to improve

Later, I decide to use build stages:

  • one stage to install the packages
  • another stage to copy the site-library/ folder.

In this way, I can write the GITHUB_PAT value in Dockerfile 2, and docker history can’t see it since the second stage doesn’t use it.

5 merging together comes perfect way

After many experiments, I find that there are two conditions to meet for exposing an ARG

  • the final stage contains ARG TEST
  • the final stage contains at least one RUN command 3

As you can see from docker history, even I just not use TEST at all), the value is still disclosed.

sha256:38e29efa3a95b241be1ad285a534e9353c2a13f2a0c5abac69ac76c5511131e9   44 minutes ago      |1 TEST=test /bin/sh -c echo haha

Finally, I figured out that after splitting into two stages, now I can pass GITHUB_PAT through --build-arg.

6 Final solution

  • Dockerfile
FROM dongzhuoer/rlang:devtools as site-library

ARG GITHUB_PAT

RUN mkdir /usr/local/lib/R/site-library/
RUN R -e "remotes::install_github(c('tidyverse/tidyverse'), lib = '/usr/local/lib/R/site-library/')"



FROM dongzhuoer/rlang:devtools

LABEL maintainer="Zhuoer Dong <dongzhuoer@mail.nankai.edu.cn>"

COPY --from=site-library /usr/local/lib/R/site-library/ /usr/lib/R/site-library/
  • Build command
docker build --build-arg GITHUB_PAT=${GITHUB_PAT} ...

The best parts of this method are:

  1. the Dockerfile is absolutely portable: anyone can still build the image 4
  2. if you set the environment variable, GITHUB_PAT, you can take advantage of higher API rate limit.

7 Epilogue

By the way, you need RUN apt update && apt -y install git && rm -r /var/lib/apt/lists/ if the R package repository is private.


  1. Dockerfile is portable, only docker build can interact with specific host machine↩︎

  2. Of course the repository containing the Dockerfile has to be private then↩︎

  3. Maybe other command also works, but COPY definitely not↩︎

  4. as long as the package repository is public↩︎