Emmanuel Kasper: How configuration is passed from the BinderHub helm chart to a running BinderHub
Context:
At $WORK I am doing a lot of datascience work around Jupyter Notebooks and their ecosystem. Right now I am setting BinderHub, which is a service to start a Jupyter Notebook from a git repo in your browser.
For setting up BinderHub I am using the BinderHub helm chart, and I was wondering how configuration changes are propagated from the BinderHub helm chart to the process running in a Kubernetes Pod.
After going through this I can say I am not right now a great fan of Helm, as it looks to me like an unnecessary, overengineered abstraction layer on top of Kubernetes manifests. Or maybe it is just that I don t want to learn the golang templating synthax. I am looking forward to testing Kustomize as an alternative, but I havn t had the chance yet.
Starting from the list of config parameters available:
Although many parameters are mentioned in the installer document, you have to go to the developer doc at https://binderhub.readthedocs.io/en/latest/reference/ref-index.html to get a whole overview.
In my case I want to set the
hostname parameter for the Gitlab Repoprovider.
This is the relevelant snippet in the developer doc:
hostname c.GitLabRepoProvider.hostname = Unicode('gitlab.com')
The host of the GitLab instance
The string c.GitLabRepoProvider.hostname here means, that the value of the hostname parameter will be loaded at the path config.GitLabRepoProvider inside a configuration file.
Using the yaml synthax this means the configuration file should contain a snippet like:
config:
GitlabRepoProvider
hostname: my-domain.com
Digging through Kubernetes constructs: Helm values files
When installing BinderHub using the provided helm chart, we can either put the configuration snippet in the config.yaml or secret.yaml helm values files.
In my case I have put the snippet in config.yaml, since the hostname is not a secret thing, I can verify with yq that it correctly set:
$ yq --raw-output '.config.GitLabRepoProvider.hostname' config.yaml
my-domain.com
How do we make sure this parameter is properly applied to our running binder processes ?
As said previouly this parameter is passed as a value file to helm ( value or -f option) in the command:
$ helm upgrade \
binderhub \
jupyterhub/binderhub \
--install \
--version=$(RELEASE) \
--create-namespace \
--namespace=binderhub \
--values secret.yaml \
--values config.yaml \
--debug
According to the helm documentation in https://helm.sh/docs/helm/helm_install/
the values file are concatenated to form a single object, and priority will be given to the last (right-most) file specified. For example, if both myvalues.yaml and override.yaml contained a key called Test , the value set in override.yaml would take precedence:
$ helm install --values myvalues.yaml --values override.yaml myredis ./redis
Digging through Kubernetes constructs: Secrets and Volumes
When helm upgrade is run the helm values of type config are stashed in a Kubernetes secret binder-secret:
https://github.com/jupyterhub/binderhub/blob/main/helm-chart/binderhub/templates/secret.yaml#L12
stringData:
- /*
Stash away relevant Helm template values for
the BinderHub Python application to read from
in binderhub_config.py.
*/
values.yaml:
- pick .Values "config" "imageBuilderType" "cors" "dind" "pink" "extraConfig" toYaml nindent 4
We can verify that our hostname is passed to our Secret:
$ kubectl get secret binder-secret -o yaml yq --raw-output '.data."values.yaml"' base64 --decode
...
GitLabRepoProvider:
hostname: my-domain.com
...
Finally a configuration file inside the Binder pod is populated from the Secret, using the Kubernetes Volume construct.
Looking at the Pod, we do see a volume called config, created from the binder-secret Secret:
$ kubectl get pod -l component=binder -o yaml grep --context 4 binder-secret
volumes:
- name: config
secret:
defaultMode: 420
secretName: binder-secret
That volume is mounted inside the pod at /etc/binderhub/config:
volumeMounts:
- mountPath: /etc/binderhub/config/
name: config
readOnly: true
Runtime verification
Looking inside our pod we see our hostname value available in a file underneath the mount point:
oc exec binder-74d9c7db95-qtp8r -- grep hostname /etc/binderhub/config/values.yaml
hostname: my-domain.com
In
The discovery of a backdoor in XZ Utils in the spring of 2024 shocked the open source community, raising critical questions about software supply chain security. This post explores whether better Debian packaging practices could have detected this threat, offering a guide to auditing packages and suggesting future improvements.
The XZ backdoor in versions 5.6.0/5.6.1 made its way briefly into many major Linux distributions such as Debian and Fedora, but luckily didn t reach that many actual users, as the backdoored releases were quickly removed thanks to the heroic diligence of
If the changes are extensive, and you want to use a LLM to help spot potential security issues, generate the report of both the upstream and Debian packaging differences in Markdown with:
To compare changes across the new and old upstream tarball, one would need to compare commits afba662b New upstream version 5.8.0 and fa1e8796 New upstream version 5.8.1 by running
With all the above tips you can now go and try to audit your own favorite package in Debian and see if it is identical with upstream, and if not, how it differs.
There is only one tiny thing that maybe a very experienced Autotools user could potentially have noticed: the
A side project I have been working on a little since last winter and
which explores extending
Like each month, have a look at the work funded by
We had 886 responses with status code 200 and 64 connections closed.
The backend terminated 64 connections while the load balancer still had
active requests directed to it.
Let's change the Load Balancer
Great! Now all the requests worked.
This is a common issue, not specific to Envoy Proxy or the setup
shown earlier. Major cloud providers have all documented it.
Even thought the success rate is 100%, the status code distribution
show some responses with status code 503. This is the case where is not
that obvious that the problem is related to idle timeout.
However, it's clear when we look the Envoy access logs:
Key Takeaways:
A new version 0.3.5 of the
Another short status update of what happened on my side last
month. Nothing stands out too much, I enjoyed doing the OSK changes
the most as that helped to improve the typing experience further. Also
doing a small bit of kernel work again was fun (still need to figure out
the 6mq's touch controller repsonsiveness though).
See below for details on the above and more: