Debian live kernel patching infra

[Herman van Rink]

Hi,

Is anyone working on live kernel patching in Debian?

I'm a bit surprised to see so little public speak about such a nice
looking feature.

I think it would be a tremendous asset for Debian to be able to offer
live kernel updates through the security infrastructure.

I get the idea that the tools to patch a kernel are stabilizing.
To make it available to anyone the Debian security team would need to
prepare a patch for each of the previous kernels and have some
infrastructure to deliver it to end users.
As the patches are available to the team the challenge would be to get a
tool set for them to make it easy/manageable.

I assume that we could distribute the patches as a deb package. Maybe
one <current name>-livepatches package which gets updated after each CVE.

I'd like to get the ball rolling on this.

I personally would be willing to help test this and donate some cash to get
this for the community.
I imagine that more businesses would be willing to chip in.

[Ben Hutchings]

Post by Herman van Rink
Hi,
Is anyone working on live kernel patching in Debian?
I'm a bit surprised to see so little public speak about such a nice
looking feature.

Not all the necessary infrastructure is even present upstream yet.  You
can load and apply patches, but it isn't yet possible to do so safely.

In order to apply live patches safely, it is necessary either to
quiesce all tasks running in the kernel (which turns out to be
impractical) or to have a transitional period where both old and new
code are in use and each task switches to using the new code only after
reaches a suitable point in execution.

Red Hat and SUSE both worked on this as part of their own live patching
systems, and this patch series is supposed to bring that work upstream:
<https://lwn.net/Articles/681486/>.  But as you can see there is still
some way to go before this can be applied.

Post by Herman van Rink
I think it would be a tremendous asset for Debian to be able to offer
live kernel updates through the security infrastructure.
I get the idea that the tools to patch a kernel are stabilizing.
To make it available to anyone the Debian security team would need to
prepare a patch for each of the previous kernels and have some
infrastructure to deliver it to end users.

I think it would be a stretch (no pun intended) to support any kernel
version older than the previous two point releases.  So if we were in a
position to do live patches in jessie now, you would be able to apply
them to these base kernel versions:

- 3.16.7-ckt25-{1,2}
- 3.16.7-ckt20-1{,+deb8u{1,2,3,4}}

but not anything older.

Post by Herman van Rink
As the patches are available to the team the challenge would be to get a
tool set for them to make it easy/manageable.
I assume that we could distribute the patches as a deb package. Maybe
one -livepatches package which gets updated after each CVE.

To the extent that I had thought about this, I was expecting live
patches to be bundled in the linux-image package.  A single extra
package (per supported flavour) of patches would also work but makes it
less likely that users install it.

Post by Herman van Rink
I'd like to get the ball rolling on this.
I personally would be willing to help test this and donate some cash to get
this for the community.
I imagine that more businesses would be willing to chip in.

I appreciate this, but I think it may still be too early to work on the
Debian integration.  Are you also willing to sponsor work on testing
and completing the upstream live patch code?

Ben.

[Herman van Rink]

Post by Herman van Rink
Hi,
Is anyone working on live kernel patching in Debian?
I'm a bit surprised to see so little public speak about such a nice
looking feature.

Not all the necessary infrastructure is even present upstream yet. You
can load and apply patches, but it isn't yet possible to do so safely.
In order to apply live patches safely, it is necessary either to
quiesce all tasks running in the kernel (which turns out to be
impractical) or to have a transitional period where both old and new
code are in use and each task switches to using the new code only after
reaches a suitable point in execution.
Red Hat and SUSE both worked on this as part of their own live patching
<https://lwn.net/Articles/681486/>. But as you can see there is still
some way to go before this can be applied.

Post by Herman van Rink
I think it would be a tremendous asset for Debian to be able to offer
live kernel updates through the security infrastructure.
I get the idea that the tools to patch a kernel are stabilizing.
To make it available to anyone the Debian security team would need to
prepare a patch for each of the previous kernels and have some
infrastructure to deliver it to end users.

I think it would be a stretch (no pun intended) to support any kernel
version older than the previous two point releases. So if we were in a
position to do live patches in jessie now, you would be able to apply
- 3.16.7-ckt25-{1,2}
- 3.16.7-ckt20-1{,+deb8u{1,2,3,4}}
but not anything older.

Sure, we must be able to come up with a sensible guideline on what users
can expect.
The postinstall script could check if the running version is patchable
and otherwise warn the user.

Post by Herman van Rink
As the patches are available to the team the challenge would be to get a
tool set for them to make it easy/manageable.
I assume that we could distribute the patches as a deb package. Maybe
one -livepatches package which gets updated after each CVE.

To the extent that I had thought about this, I was expecting live
patches to be bundled in the linux-image package. A single extra
package (per supported flavour) of patches would also work but makes it
less likely that users install it.

Sure, I was worried that a single package might get too large/complex...
whatever is easiest to maintain.

Post by Herman van Rink
I'd like to get the ball rolling on this.
I personally would be willing to help test this and donate some cash to get
this for the community.
I imagine that more businesses would be willing to chip in.

I appreciate this, but I think it may still be too early to work on the
Debian integration.

I agree that a good consistency model is essential, but it should not
stop us from already planning for the needed Debian integration.

Are you also willing to sponsor work on testing
and completing the upstream live patch code?

Is there a tip jar?