Discussion:
[openstack-dev] [puppet] Supporting two openstack releases in the same puppet release
Andrew Woodward
2015-07-09 23:06:35 UTC
Permalink
It has come to my attention that we spend an enormous amount of time in
Fuel when switching between OpenStack releases. A good amount of this seems
to be due to a chicken and the egg problem. We cant deploy updated packages
with out updated manifests, and we we cant update (in that CI won't pass)
the manifests without updated packages. In order to accommodate this we
either have to set up a new separate set of CI pipelines for both or freeze
various parts of CI, merging patch sets that can't pass until everything is
in and working, or some monstrous combination of both. This has become
quite the time sink for everyone involved and there must be a better way to
accommodate switching between releases.

Thinking about this I wonder how much easier this process could be if the
manifests supported two releases at once.

I propose that we consider supporting both the dev release, and the prior
release (as stable and default for option precedence). We would need to
come up with some option versioning scheme in which we can identify options
which are only relevant to the specific release (I'm not yet sure what sure
what this would look line) This seems like it would be most of the need to
support multiple versions at once.

But why should we go through this effort? Yes this would become complicated
to set up, however I cant imagine that other folks are not having a similar
issues.

* Moving to a two version system should nearly reduce the impact of
switching to nearly none (given you have packages)
* It should also help accelerate our releases of puppet-openstack, since
devs can switch between versions more easily
* It should help our release support story since non-option changes will
automatically hit dev and (current) stable manifests at the same time
* This can also help the current stable release process, instead of
branching, we can simply tag the first stable release and maintain it in
the master branch, only pushing it out when we drop it to shift the
dev/stable versions
* While not a direct benefit to puppet-openstack, this would help simplify
fuel running CI on master openstack and master puppet-openstack which will
help with our efforts in puppet-openstack

So next steps
Lets discuss the proposal in general (good idea / bad idea). I'd like to
hash out what this versioned interface may need to look like, that way I
can take up attempting to make this work in a single module so we can
vet/discuss any issues more thoroughly.
--
--

Andrew Woodward

Mirantis

Fuel Community Ambassador

Ceph Community
Emilien Macchi
2015-07-10 04:57:18 UTC
Permalink
Post by Andrew Woodward
It has come to my attention that we spend an enormous amount of time in
Fuel when switching between OpenStack releases. A good amount of this
seems to be due to a chicken and the egg problem. We cant deploy updated
packages with out updated manifests, and we we cant update (in that CI
won't pass) the manifests without updated packages. In order to
accommodate this we either have to set up a new separate set of CI
pipelines for both or freeze various parts of CI, merging patch sets
that can't pass until everything is in and working, or some monstrous
combination of both. This has become quite the time sink for everyone
involved and there must be a better way to accommodate switching between
releases.
"We can't (...), and we can't (...)." :
This is a matter of orchestration and I think the module composition has
nothing to do/help with upgrade support.

Since I've been working on OpenStack, I contributed to two Open-Source
installers, and both were able to upgrade OpenStack using *upstream*
Puppet modules.

For example, Spinal Stack is running image-based deployment, that means
we don't upgrade packages on all nodes (it does not scale and it's not
safe, but this is out of topic here). We also use Puppetfile and r10k to
upgrade modules on our puppet client nodes (we run masterless). There is
no magic here, we use Ansible to do that things.
Post by Andrew Woodward
Thinking about this I wonder how much easier this process could be if
the manifests supported two releases at once.
Please define "support".
To me, you can deploy current master on Kilo, even though master targets
Liberty right now.

If you missed that, we had a whole dedicated session about that during
the last Summit, and here is the blueprint that our group is writing:
https://review.openstack.org/180141

Feel free to be part of the discussion!

Another thing is, we are currently working on testing upgrade between
OpenStack releases. You probably noticed, but we have a CI job that is
called "gate-puppet-*-puppet-beaker-rspec-dsvm-*".
It basically aims to checkout stable/kilo, run acceptance (beaker will
deploy OpenStack and create OpenStack resources), checkout the patchset
in sent in master, run acceptance again (beaker will upgrade OpenStack
and test if resources survived to the upgrade). This is a `grenade` like.
Post by Andrew Woodward
I propose that we consider supporting both the dev release, and the
prior release (as stable and default for option precedence). We would
need to come up with some option versioning scheme in which we can
identify options which are only relevant to the specific release (I'm
not yet sure what sure what this would look line) This seems like it
would be most of the need to support multiple versions at once.
Is your intention to open stable branches to multiple release support?
It sounds hard and expensive to manage in term of maintenance & release.
Post by Andrew Woodward
But why should we go through this effort? Yes this would become
complicated to set up, however I cant imagine that other folks are not
having a similar issues.
* Moving to a two version system should nearly reduce the impact of
switching to nearly none (given you have packages)
Does OpenStack work like this? Can you install Nova from the same branch
and run Kilo/Liberty?
Post by Andrew Woodward
* It should also help accelerate our releases of puppet-openstack, since
devs can switch between versions more easily
It will be a mess to maintain and manage.
Post by Andrew Woodward
* It should help our release support story since non-option changes will
automatically hit dev and (current) stable manifests at the same time
I don't know how it will help, really. Maybe can you give some examples
here at the difficulties we have today and how you can fix it with
having 2 same releases in one branch.
Post by Andrew Woodward
* This can also help the current stable release process, instead of
branching, we can simply tag the first stable release and maintain it in
the master branch, only pushing it out when we drop it to shift the
dev/stable versions
* While not a direct benefit to puppet-openstack, this would help
simplify fuel running CI on master openstack and master puppet-openstack
which will help with our efforts in puppet-openstack
I have the feeling you're trying to make something that will arrange
Fuel project while it breaks the whole Puppet OpenStack model we're
building, that tries to follow other OpenStack projects to be more
consistent.
I'm very happy if Fuel team is pushing some efforts in Puppet OpenStack
(and even more lately), but one thing is sure, we won't change our model
for Fuel.
Post by Andrew Woodward
So next steps
Lets discuss the proposal in general (good idea / bad idea). I'd like to
hash out what this versioned interface may need to look like, that way I
can take up attempting to make this work in a single module so we can
vet/discuss any issues more thoroughly.
We have around 20 Puppet modules with stable branches. Are you sure to
have the bandwidth to do such a work? It sounds really huge and I'm not
sure to see a real interest.

I'm also curious to read from others.
Post by Andrew Woodward
--
Andrew Woodward
Mirantis
Fuel Community Ambassador
Ceph Community
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
--
Emilien Macchi
Loading...