Discussion:
[openstack-dev] [all][infra] Upcoming changes in ARA Zuul job reports
David Moreau Simard
2018-03-29 22:14:06 UTC
Permalink
Hi,

By default, all jobs currently benefit from the generation of a static
ARA report located in the "ara" directory at the root of the log
directory.
Due to scalability concerns, these reports were only generated when a
job failed and were not available on successful runs.

I'm happy to announce that you can expect ARA reports to be available
for every job from now on -- including the successful ones !

You'll notice a subtle but important change: the report directory will
henceforth be named "ara-report" instead of "ara".

Instead of generating and saving a HTML report, we'll now only save
the ARA database in the "ara-report" directory.
This is a special directory from the perspective of the
logs.openstack.org server and ARA databases located in such
directories will be loaded dynamically by a WSGI middleware.

You don't need to do anything to benefit from this change -- it will
be pushed to all jobs that inherit from the base job by default.

However, if you happen to be using a "nested" installation of ARA and
Ansible (i.e, OpenStack-Ansible, Kolla-Ansible, TripleO, etc.), this
means that you can also leverage this feature.
In order to do that, you'll want to create an "ara-report" directory
and copy your ARA database inside before your logs are collected and
uploaded.

To help you visualize:
<logroot>/ara-report <-- This is the default Zuul report
<logroot>/logs/ara <-- This wouldn't be loaded dynamically
<logroot>/logs/ara-report <-- This would be loaded dynamically
<logroot>/logs/some/directory/ara-report <-- This would be loaded dynamically

For more details on this feature of ARA, you can refer to the documentation [1].

Let me know if you have any questions !

[1]: https://ara.readthedocs.io/en/latest/advanced.html

David Moreau Simard
Senior Software Engineer | OpenStack RDO

dmsimard = [irc, github, twitter]

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-***@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/l
Paul Belanger
2018-03-29 23:12:35 UTC
Permalink
Post by David Moreau Simard
Hi,
By default, all jobs currently benefit from the generation of a static
ARA report located in the "ara" directory at the root of the log
directory.
Due to scalability concerns, these reports were only generated when a
job failed and were not available on successful runs.
I'm happy to announce that you can expect ARA reports to be available
for every job from now on -- including the successful ones !
You'll notice a subtle but important change: the report directory will
henceforth be named "ara-report" instead of "ara".
Instead of generating and saving a HTML report, we'll now only save
the ARA database in the "ara-report" directory.
This is a special directory from the perspective of the
logs.openstack.org server and ARA databases located in such
directories will be loaded dynamically by a WSGI middleware.
You don't need to do anything to benefit from this change -- it will
be pushed to all jobs that inherit from the base job by default.
However, if you happen to be using a "nested" installation of ARA and
Ansible (i.e, OpenStack-Ansible, Kolla-Ansible, TripleO, etc.), this
means that you can also leverage this feature.
In order to do that, you'll want to create an "ara-report" directory
and copy your ARA database inside before your logs are collected and
uploaded.
I believe this is an important task we should also push on for the projects you
listed above. The main reason to do this is simplify job uploads and filesystemd
demands (thanks clarkb).

Lets see if we can update these projects in the coming week or two!

Great work.
Post by David Moreau Simard
<logroot>/ara-report <-- This is the default Zuul report
<logroot>/logs/ara <-- This wouldn't be loaded dynamically
<logroot>/logs/ara-report <-- This would be loaded dynamically
<logroot>/logs/some/directory/ara-report <-- This would be loaded dynamically
For more details on this feature of ARA, you can refer to the documentation [1].
Let me know if you have any questions !
[1]: https://ara.readthedocs.io/en/latest/advanced.html
David Moreau Simard
Senior Software Engineer | OpenStack RDO
dmsimard = [irc, github, twitter]
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-***@lists.openstack.org?subject:unsubscribe
http://lis
Jeffrey Zhang
2018-03-30 01:05:23 UTC
Permalink
cool. kolla will try to implement it.
Post by Paul Belanger
Post by David Moreau Simard
Hi,
By default, all jobs currently benefit from the generation of a static
ARA report located in the "ara" directory at the root of the log
directory.
Due to scalability concerns, these reports were only generated when a
job failed and were not available on successful runs.
I'm happy to announce that you can expect ARA reports to be available
for every job from now on -- including the successful ones !
You'll notice a subtle but important change: the report directory will
henceforth be named "ara-report" instead of "ara".
Instead of generating and saving a HTML report, we'll now only save
the ARA database in the "ara-report" directory.
This is a special directory from the perspective of the
logs.openstack.org server and ARA databases located in such
directories will be loaded dynamically by a WSGI middleware.
You don't need to do anything to benefit from this change -- it will
be pushed to all jobs that inherit from the base job by default.
However, if you happen to be using a "nested" installation of ARA and
Ansible (i.e, OpenStack-Ansible, Kolla-Ansible, TripleO, etc.), this
means that you can also leverage this feature.
In order to do that, you'll want to create an "ara-report" directory
and copy your ARA database inside before your logs are collected and
uploaded.
I believe this is an important task we should also push on for the projects you
listed above. The main reason to do this is simplify job uploads and filesystemd
demands (thanks clarkb).
Lets see if we can update these projects in the coming week or two!
Great work.
Post by David Moreau Simard
<logroot>/ara-report <-- This is the default Zuul report
<logroot>/logs/ara <-- This wouldn't be loaded dynamically
<logroot>/logs/ara-report <-- This would be loaded dynamically
<logroot>/logs/some/directory/ara-report <-- This would be loaded
dynamically
Post by David Moreau Simard
For more details on this feature of ARA, you can refer to the
documentation [1].
Post by David Moreau Simard
Let me know if you have any questions !
[1]: https://ara.readthedocs.io/en/latest/advanced.html
David Moreau Simard
Senior Software Engineer | OpenStack RDO
dmsimard = [irc, github, twitter]
____________________________________________________________
______________
Post by David Moreau Simard
OpenStack Development Mailing List (not for usage questions)
unsubscribe
Post by David Moreau Simard
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
--
Regards,
Jeffrey Zhang
Blog: http://xcodest.me
David Moreau Simard
2018-04-03 17:00:47 UTC
Permalink
Post by Jeffrey Zhang
cool. kolla will try to implement it.
Cool !
For reference, openstack-ansible already retooled their log collection
to copy the database instead of generating the report [1].

[1]: https://review.openstack.org/#/c/557921/

David Moreau Simard
Senior Software Engineer | OpenStack RDO

dmsimard = [irc, github, twitter]

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-***@lists.openstack.org?subject:unsubscribe
http://lists.
David Moreau Simard
2018-04-04 22:16:15 UTC
Permalink
Hi,

You might have noticed that the performance (and reliability) of the
new reports aren't up to par.
If you see failures in loading content, a refresh will usually fix the issue.

We have different fixes to improve the performance and the reliability
of the reports and we hope
to be able to land them soon.

In the meantime, please let us know if there is any report that
appears to be particularly
problematic.

Thanks !


David Moreau Simard
Senior Software Engineer | OpenStack RDO

dmsimard = [irc, github, twitter]


On Thu, Mar 29, 2018 at 6:14 PM, David Moreau Simard
Post by David Moreau Simard
Hi,
By default, all jobs currently benefit from the generation of a static
ARA report located in the "ara" directory at the root of the log
directory.
Due to scalability concerns, these reports were only generated when a
job failed and were not available on successful runs.
I'm happy to announce that you can expect ARA reports to be available
for every job from now on -- including the successful ones !
You'll notice a subtle but important change: the report directory will
henceforth be named "ara-report" instead of "ara".
Instead of generating and saving a HTML report, we'll now only save
the ARA database in the "ara-report" directory.
This is a special directory from the perspective of the
logs.openstack.org server and ARA databases located in such
directories will be loaded dynamically by a WSGI middleware.
You don't need to do anything to benefit from this change -- it will
be pushed to all jobs that inherit from the base job by default.
However, if you happen to be using a "nested" installation of ARA and
Ansible (i.e, OpenStack-Ansible, Kolla-Ansible, TripleO, etc.), this
means that you can also leverage this feature.
In order to do that, you'll want to create an "ara-report" directory
and copy your ARA database inside before your logs are collected and
uploaded.
<logroot>/ara-report <-- This is the default Zuul report
<logroot>/logs/ara <-- This wouldn't be loaded dynamically
<logroot>/logs/ara-report <-- This would be loaded dynamically
<logroot>/logs/some/directory/ara-report <-- This would be loaded dynamically
For more details on this feature of ARA, you can refer to the documentation [1].
Let me know if you have any questions !
[1]: https://ara.readthedocs.io/en/latest/advanced.html
David Moreau Simard
Senior Software Engineer | OpenStack RDO
dmsimard = [irc, github, twitter]
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-***@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailm
David Moreau Simard
2018-05-04 19:22:26 UTC
Permalink
Hi,

It took longer than ancitipated but the necessary changes have landed
in both ARA and logs.openstack.org.
The reports should now be faster and more reliable.

Please let me know if you see anything out of the ordinary.

Thanks !


David Moreau Simard
Senior Software Engineer | OpenStack RDO

dmsimard = [irc, github, twitter]
Post by David Moreau Simard
Hi,
You might have noticed that the performance (and reliability) of the
new reports aren't up to par.
If you see failures in loading content, a refresh will usually fix the issue.
We have different fixes to improve the performance and the reliability
of the reports and we hope
to be able to land them soon.
In the meantime, please let us know if there is any report that
appears to be particularly
problematic.
Thanks !
David Moreau Simard
Senior Software Engineer | OpenStack RDO
dmsimard = [irc, github, twitter]
On Thu, Mar 29, 2018 at 6:14 PM, David Moreau Simard
Post by David Moreau Simard
Hi,
By default, all jobs currently benefit from the generation of a static
ARA report located in the "ara" directory at the root of the log
directory.
Due to scalability concerns, these reports were only generated when a
job failed and were not available on successful runs.
I'm happy to announce that you can expect ARA reports to be available
for every job from now on -- including the successful ones !
You'll notice a subtle but important change: the report directory will
henceforth be named "ara-report" instead of "ara".
Instead of generating and saving a HTML report, we'll now only save
the ARA database in the "ara-report" directory.
This is a special directory from the perspective of the
logs.openstack.org server and ARA databases located in such
directories will be loaded dynamically by a WSGI middleware.
You don't need to do anything to benefit from this change -- it will
be pushed to all jobs that inherit from the base job by default.
However, if you happen to be using a "nested" installation of ARA and
Ansible (i.e, OpenStack-Ansible, Kolla-Ansible, TripleO, etc.), this
means that you can also leverage this feature.
In order to do that, you'll want to create an "ara-report" directory
and copy your ARA database inside before your logs are collected and
uploaded.
<logroot>/ara-report <-- This is the default Zuul report
<logroot>/logs/ara <-- This wouldn't be loaded dynamically
<logroot>/logs/ara-report <-- This would be loaded dynamically
<logroot>/logs/some/directory/ara-report <-- This would be loaded dynamically
For more details on this feature of ARA, you can refer to the documentation [1].
Let me know if you have any questions !
[1]: https://ara.readthedocs.io/en/latest/advanced.html
David Moreau Simard
Senior Software Engineer | OpenStack RDO
dmsimard = [irc, github, twitter]
Loading...