Discussion:
[horizon] REST and Django
(too old to reply)
Tripp, Travis S
2014-11-27 18:56:47 UTC
Permalink
Hi Richard,

You are right, we should put this out on the main ML, so copying thread out to there. ML: FYI that this started after some impromptu IRC discussions about a specific patch led into an impromptu google hangout discussion with all the people on the thread below.

As I mentioned in the review[1], Thai and I were mainly discussing the possible performance implications of network hops from client to horizon server and whether or not any aggregation should occur server side. In other words, some views require several APIs to be queried before any data can displayed and it would eliminate some extra network requests from client to server if some of the data was first collected on the server side across service APIs. For example, the launch instance wizard will need to collect data from quite a few APIs before even the first step is displayed (I’ve listed those out in the blueprint [2]).

The flip side to that (as you also pointed out) is that if we keep the API’s fine grained then the wizard will be able to optimize in one place the calls for data as it is needed. For example, the first step may only need half of the API calls. It also could lead to perceived performance increases just due to the wizard making a call for different data independently and displaying it as soon as it can.

I tend to lean towards your POV and starting with discrete API calls and letting the client optimize calls. If there are performance problems or other reasons then doing data aggregation on the server side could be considered at that point. Of course if anybody is able to do some performance testing between the two approaches then that could affect the direction taken.

[1] https://review.openstack.org/#/c/136676/8/openstack_dashboard/api/rest/urls.py
[2] https://blueprints.launchpad.net/horizon/+spec/launch-instance-redesign

-Travis

From: Richard Jones <***@gmail.com<mailto:***@gmail.com>>
Date: Wednesday, November 26, 2014 at 11:55 PM
To: Travis Tripp <***@hp.com<mailto:***@hp.com>>, Thai Q Tran/Silicon Valley/IBM <***@us.ibm.com<mailto:***@us.ibm.com>>, David Lyle <***@gmail.com<mailto:***@gmail.com>>, Maxime Vidori <***@enovance.com<mailto:***@enovance.com>>, "Wroblewski, Szymon" <***@intel.com<mailto:***@intel.com>>, "Wood, Matthew David (HP Cloud - Horizon)" <***@hp.com<mailto:***@hp.com>>, "Chen, Shaoquan" <***@hp.com<mailto:***@hp.com>>, "Farina, Matt (HP Cloud)" <***@hp.com<mailto:***@hp.com>>, Cindy Lu/Silicon Valley/IBM <***@us.ibm.com<mailto:***@us.ibm.com>>, Justin Pomeroy/Rochester/IBM <***@us.ibm.com<mailto:***@us.ibm.com>>, Neill Cox <***@ingenious.com.au<mailto:***@ingenious.com.au>>
Subject: Re: REST and Django

I'm not sure whether this is the appropriate place to discuss this, or whether I should be posting to the list under [Horizon] but I think we need to have a clear idea of what goes in the REST API and what goes in the client (angular) code.

In my mind, the thinner the REST API the better. Indeed if we can get away with proxying requests through without touching any *client code, that would be great.

Coding additional logic into the REST API means that a developer would need to look in two places, instead of one, to determine what was happening for a particular call. If we keep it thin then the API presented to the client developer is very, very similar to the API presented by the services. Minimum surprise.

Your thoughts?


Richard


On Wed Nov 26 2014 at 2:40:52 PM Richard Jones <***@gmail.com<mailto:***@gmail.com>> wrote:
Thanks for the great summary, Travis.

I've completed the work I pledged this morning, so now the REST API change set has:

- no rest framework dependency
- AJAX scaffolding in openstack_dashboard.api.rest.utils
- code in openstack_dashboard/api/rest/
- renamed the API from "identity" to "keystone" to be consistent
- added a sample of testing, mostly for my own sanity to check things were working

https://review.openstack.org/#/c/136676


Richard

On Wed Nov 26 2014 at 12:18:25 PM Tripp, Travis S <***@hp.com<mailto:***@hp.com>> wrote:
Hello all,

Great discussion on the REST urls today! I think that we are on track to come to a common REST API usage pattern. To provide quick summary:

We all agreed that going to a straight REST pattern rather than through tables was a good idea. We discussed using direct get / post in Django views like what Max originally used[1][2] and Thai also started[3] with the identity table rework or to go with djangorestframework [5] like what Richard was prototyping with[4].

The main things we would use from Django Rest Framework were built in JSON serialization (avoid boilerplate), better exception handling, and some request wrapping. However, we all weren’t sure about the need for a full new framework just for that. At the end of the conversation, we decided that it was a cleaner approach, but Richard would see if he could provide some utility code to do that much for us without requiring the full framework. David voiced that he doesn’t want us building out a whole framework on our own either.

So, Richard will do some investigation during his day today and get back to us. Whatever the case, we’ll get a patch in horizon for the base dependency (framework or Richard’s utilities) that both Thai’s work and the launch instance work is dependent upon. We’ll build REST style API’s using the same pattern. We will likely put the rest api’s in horizon/openstack_dashboard/api/rest/.

[1] https://review.openstack.org/#/c/133178/1/openstack_dashboard/workflow/keypair.py
[2] https://review.openstack.org/#/c/133178/1/openstack_dashboard/workflow/launch.py
[3] https://review.openstack.org/#/c/133767/8/openstack_dashboard/dashboards/identity/users/views.py
[4] https://review.openstack.org/#/c/136676/4/openstack_dashboard/rest_api/identity.py
[5] http://www.django-rest-framework.org/

Thanks,
Travis
Richard Jones
2014-11-28 01:38:01 UTC
Permalink
On Fri Nov 28 2014 at 5:58:00 AM Tripp, Travis S <***@hp.com>
wrote:

> Hi Richard,
>
> You are right, we should put this out on the main ML, so copying thread
> out to there. ML: FYI that this started after some impromptu IRC
> discussions about a specific patch led into an impromptu google hangout
> discussion with all the people on the thread below.
>

Thanks Travis!



> As I mentioned in the review[1], Thai and I were mainly discussing the
> possible performance implications of network hops from client to horizon
> server and whether or not any aggregation should occur server side. In
> other words, some views require several APIs to be queried before any data
> can displayed and it would eliminate some extra network requests from
> client to server if some of the data was first collected on the server side
> across service APIs. For example, the launch instance wizard will need to
> collect data from quite a few APIs before even the first step is displayed
> (I’ve listed those out in the blueprint [2]).
>
> The flip side to that (as you also pointed out) is that if we keep the
> API’s fine grained then the wizard will be able to optimize in one place
> the calls for data as it is needed. For example, the first step may only
> need half of the API calls. It also could lead to perceived performance
> increases just due to the wizard making a call for different data
> independently and displaying it as soon as it can.
>

Indeed, looking at the current launch wizard code it seems like you
wouldn't need to load all that data for the wizard to be displayed, since
only some subset of it would be necessary to display any given panel of the
wizard.



> I tend to lean towards your POV and starting with discrete API calls and
> letting the client optimize calls. If there are performance problems or
> other reasons then doing data aggregation on the server side could be
> considered at that point.
>

I'm glad to hear it. I'm a fan of optimising when necessary, and not
beforehand :)



> Of course if anybody is able to do some performance testing between the
> two approaches then that could affect the direction taken.
>

I would certainly like to see us take some measurements when performance
issues pop up. Optimising without solid metrics is bad idea :)


Richard


>
> [1]
> https://review.openstack.org/#/c/136676/8/openstack_dashboard/api/rest/urls.py
> [2]
> https://blueprints.launchpad.net/horizon/+spec/launch-instance-redesign
>
> -Travis
>
> From: Richard Jones <***@gmail.com>
> Date: Wednesday, November 26, 2014 at 11:55 PM
> To: Travis Tripp <***@hp.com>, Thai Q Tran/Silicon Valley/IBM <
> ***@us.ibm.com>, David Lyle <***@gmail.com>, Maxime Vidori <
> ***@enovance.com>, "Wroblewski, Szymon" <
> ***@intel.com>, "Wood, Matthew David (HP Cloud - Horizon)" <
> ***@hp.com>, "Chen, Shaoquan" <***@hp.com>, "Farina, Matt
> (HP Cloud)" <***@hp.com>, Cindy Lu/Silicon Valley/IBM <
> ***@us.ibm.com>, Justin Pomeroy/Rochester/IBM <***@us.ibm.com>, Neill
> Cox <***@ingenious.com.au>
> Subject: Re: REST and Django
>
> I'm not sure whether this is the appropriate place to discuss this, or
> whether I should be posting to the list under [Horizon] but I think we need
> to have a clear idea of what goes in the REST API and what goes in the
> client (angular) code.
>
> In my mind, the thinner the REST API the better. Indeed if we can get
> away with proxying requests through without touching any *client code, that
> would be great.
>
> Coding additional logic into the REST API means that a developer would
> need to look in two places, instead of one, to determine what was happening
> for a particular call. If we keep it thin then the API presented to the
> client developer is very, very similar to the API presented by the
> services. Minimum surprise.
>
> Your thoughts?
>
>
> Richard
>
>
> On Wed Nov 26 2014 at 2:40:52 PM Richard Jones <***@gmail.com>
> wrote:
>
>> Thanks for the great summary, Travis.
>>
>> I've completed the work I pledged this morning, so now the REST API
>> change set has:
>>
>> - no rest framework dependency
>> - AJAX scaffolding in openstack_dashboard.api.rest.utils
>> - code in openstack_dashboard/api/rest/
>> - renamed the API from "identity" to "keystone" to be consistent
>> - added a sample of testing, mostly for my own sanity to check things
>> were working
>>
>> https://review.openstack.org/#/c/136676
>>
>>
>> Richard
>>
>> On Wed Nov 26 2014 at 12:18:25 PM Tripp, Travis S <***@hp.com>
>> wrote:
>>
>>> Hello all,
>>>
>>> Great discussion on the REST urls today! I think that we are on track
>>> to come to a common REST API usage pattern. To provide quick summary:
>>>
>>> We all agreed that going to a straight REST pattern rather than
>>> through tables was a good idea. We discussed using direct get / post in
>>> Django views like what Max originally used[1][2] and Thai also started[3]
>>> with the identity table rework or to go with djangorestframework [5] like
>>> what Richard was prototyping with[4].
>>>
>>> The main things we would use from Django Rest Framework were built in
>>> JSON serialization (avoid boilerplate), better exception handling, and some
>>> request wrapping. However, we all weren’t sure about the need for a full
>>> new framework just for that. At the end of the conversation, we decided
>>> that it was a cleaner approach, but Richard would see if he could provide
>>> some utility code to do that much for us without requiring the full
>>> framework. David voiced that he doesn’t want us building out a whole
>>> framework on our own either.
>>>
>>> So, Richard will do some investigation during his day today and get
>>> back to us. Whatever the case, we’ll get a patch in horizon for the base
>>> dependency (framework or Richard’s utilities) that both Thai’s work and the
>>> launch instance work is dependent upon. We’ll build REST style API’s using
>>> the same pattern. We will likely put the rest api’s in
>>> horizon/openstack_dashboard/api/rest/.
>>>
>>> [1] https://review.openstack.org/#/c/133178/1/openstack_
>>> dashboard/workflow/keypair.py
>>> [2] https://review.openstack.org/#/c/133178/1/openstack_
>>> dashboard/workflow/launch.py
>>> [3] https://review.openstack.org/#/c/133767/8/openstack_
>>> dashboard/dashboards/identity/users/views.py
>>> [4] https://review.openstack.org/#/c/136676/4/openstack_
>>> dashboard/rest_api/identity.py
>>> [5] http://www.django-rest-framework.org/
>>>
>>> Thanks,
>>> Travis
>>>
>>
Thai Q Tran
2014-12-01 05:17:28 UTC
Permalink
I agree that keeping the API layer thin would be ideal. I should add that
having discrete API calls would allow dynamic population of table. However,
I will make a case where it might be necessary to add additional APIs.
Consider that you want to delete 3 items in a given table.

If you do this on the client side, you would need to perform: n * (1 API
request + 1 AJAX request)
If you have some logic on the server side that batch delete actions: n * (1
API request) + 1 AJAX request

Consider the following:
n = 1, client = 2 trips, server = 2 trips
n = 3, client = 6 trips, server = 4 trips
n = 10, client = 20 trips, server = 11 trips
n = 100, client = 200 trips, server 101 trips

As you can see, this does not scale very well.... something to consider...




From: Richard Jones <***@gmail.com>
To: "Tripp, Travis S" <***@hp.com>, OpenStack List
<openstack-***@lists.openstack.org>
Date: 11/27/2014 05:38 PM
Subject: Re: [openstack-dev] [horizon] REST and Django



On Fri Nov 28 2014 at 5:58:00 AM Tripp, Travis S <***@hp.com>
wrote:
Hi Richard,

You are right, we should put this out on the main ML, so copying thread
out to there.  ML: FYI that this started after some impromptu IRC
discussions about a specific patch led into an impromptu google hangout
discussion with all the people on the thread below.

Thanks Travis!


As I mentioned in the review[1], Thai and I were mainly discussing the
possible performance implications of network hops from client to horizon
server and whether or not any aggregation should occur server side.   In
other words, some views  require several APIs to be queried before any
data can displayed and it would eliminate some extra network requests
from client to server if some of the data was first collected on the
server side across service APIs.  For example, the launch instance wizard
will need to collect data from quite a few APIs before even the first
step is displayed (I’ve listed those out in the blueprint [2]).

The flip side to that (as you also pointed out) is that if we keep the
API’s fine grained then the wizard will be able to optimize in one place
the calls for data as it is needed. For example, the first step may only
need half of the API calls. It also could lead to perceived performance
increases just due to the wizard making a call for different data
independently and displaying it as soon as it can.

Indeed, looking at the current launch wizard code it seems like you
wouldn't need to load all that data for the wizard to be displayed, since
only some subset of it would be necessary to display any given panel of the
wizard.


I tend to lean towards your POV and starting with discrete API calls and
letting the client optimize calls.  If there are performance problems or
other reasons then doing data aggregation on the server side could be
considered at that point.

I'm glad to hear it. I'm a fan of optimising when necessary, and not
beforehand :)


Of course if anybody is able to do some performance testing between the
two approaches then that could affect the direction taken.

I would certainly like to see us take some measurements when performance
issues pop up. Optimising without solid metrics is bad idea :)


    Richard


[1]
https://review.openstack.org/#/c/136676/8/openstack_dashboard/api/rest/urls.py
[2]
https://blueprints.launchpad.net/horizon/+spec/launch-instance-redesign

-Travis

From: Richard Jones <***@gmail.com>
Date: Wednesday, November 26, 2014 at 11:55 PM
To: Travis Tripp <***@hp.com>, Thai Q Tran/Silicon Valley/IBM <
***@us.ibm.com>, David Lyle <***@gmail.com>, Maxime Vidori <
***@enovance.com>, "Wroblewski, Szymon" <
***@intel.com>, "Wood, Matthew David (HP Cloud - Horizon)"
<***@hp.com>, "Chen, Shaoquan" <***@hp.com>, "Farina, Matt
(HP Cloud)" <***@hp.com>, Cindy Lu/Silicon Valley/IBM <
***@us.ibm.com>, Justin Pomeroy/Rochester/IBM <***@us.ibm.com>, Neill
Cox <***@ingenious.com.au>
Subject: Re: REST and Django

I'm not sure whether this is the appropriate place to discuss this, or
whether I should be posting to the list under [Horizon] but I think we
need to have a clear idea of what goes in the REST API and what goes in
the client (angular) code.

In my mind, the thinner the REST API the better. Indeed if we can get
away with proxying requests through without touching any *client code,
that would be great.

Coding additional logic into the REST API means that a developer would
need to look in two places, instead of one, to determine what was
happening for a particular call. If we keep it thin then the API
presented to the client developer is very, very similar to the API
presented by the services. Minimum surprise.

Your thoughts?


     Richard


On Wed Nov 26 2014 at 2:40:52 PM Richard Jones <***@gmail.com>
wrote:
Thanks for the great summary, Travis.

I've completed the work I pledged this morning, so now the REST API
change set has:

- no rest framework dependency
- AJAX scaffolding in openstack_dashboard.api.rest.utils
- code in openstack_dashboard/api/rest/
- renamed the API from "identity" to "keystone" to be consistent
- added a sample of testing, mostly for my own sanity to check things
were working

https://review.openstack.org/#/c/136676


      Richard

On Wed Nov 26 2014 at 12:18:25 PM Tripp, Travis S <***@hp.com>
wrote:
Hello all,

Great discussion on the REST urls today! I think that we are on track
to come to a common REST API usage pattern.  To provide quick summary:

We all agreed that going to a straight REST pattern rather than
through tables was a good idea. We discussed using direct get / post
in Django views like what Max originally used[1][2] and Thai also
started[3] with the identity table rework or to go with
djangorestframework [5] like what Richard was prototyping with[4].

The main things we would use from Django Rest Framework were built in
JSON serialization (avoid boilerplate), better exception handling, and
some request wrapping.  However, we all weren’t sure about the need
for a full new framework just for that. At the end of the
conversation, we decided that it was a cleaner approach, but Richard
would see if he could provide some utility code to do that much for us
without requiring the full framework.  David voiced that he doesn’t
want us building out a whole framework on our own either.

So, Richard will do some investigation during his day today and get
back to us.  Whatever the case, we’ll get a patch in horizon for the
base dependency (framework or Richard’s utilities) that both Thai’s
work and the launch instance work is dependent upon.  We’ll build REST
style API’s using the same pattern.  We will likely put the rest api’s
in horizon/openstack_dashboard/api/rest/.

[1]
https://review.openstack.org/#/c/133178/1/openstack_dashboard/workflow/keypair.py
[2]
https://review.openstack.org/#/c/133178/1/openstack_dashboard/workflow/launch.py
[3]
https://review.openstack.org/#/c/133767/8/openstack_dashboard/dashboards/identity/users/views.py
[4]
https://review.openstack.org/#/c/136676/4/openstack_dashboard/rest_api/identity.py
[5] http://www.django-rest-framework.org/

Thanks,
Travis_______________________________________________
OpenStack-dev mailing list
OpenStack-***@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Wood, Matthew David (HP Cloud - Horizon)
2014-12-01 15:31:25 UTC
Permalink
In theory, for many cases, the service(s) will allow this to happen with ~1 rest call. I THINK this was a big part of the batch action code, atleast at the beginning, though I think we’ve (unfortunately) started moving away from that idea.
--
Matthew Wood
HP Cloud Services
Full-Stack Engineer
Python Lover
***@hp.com
303.818.7497

From: Thai Q Tran <***@us.ibm.com><mailto:***@us.ibm.com>
Reply: OpenStack Development Mailing List (not for usage questions) <openstack-***@lists.openstack.org>><mailto:openstack-***@lists.openstack.org>
Date: November 30, 2014 at 10:20:29 PM
To: OpenStack Development Mailing List (not for usage questions) <openstack-***@lists.openstack.org>><mailto:openstack-***@lists.openstack.org>
Subject: Re: [openstack-dev] [horizon] REST and Django


I agree that keeping the API layer thin would be ideal. I should add that having discrete API calls would allow dynamic population of table. However, I will make a case where it might be necessary to add additional APIs. Consider that you want to delete 3 items in a given table.

If you do this on the client side, you would need to perform: n * (1 API request + 1 AJAX request)
If you have some logic on the server side that batch delete actions: n * (1 API request) + 1 AJAX request

Consider the following:
n = 1, client = 2 trips, server = 2 trips
n = 3, client = 6 trips, server = 4 trips
n = 10, client = 20 trips, server = 11 trips
n = 100, client = 200 trips, server 101 trips

As you can see, this does not scale very well.... something to consider...


[Inactive hide details for Richard Jones ---11/27/2014 05:38:53 PM---On Fri Nov 28 2014 at 5:58:00 AM Tripp, Travis S <travis.tr] Richard Jones ---11/27/2014 05:38:53 PM---On Fri Nov 28 2014 at 5:58:00 AM Tripp, Travis S <***@hp.com> wrote:

From: Richard Jones <***@gmail.com>
To: "Tripp, Travis S" <***@hp.com>, OpenStack List <openstack-***@lists.openstack.org>
Date: 11/27/2014 05:38 PM
Subject: Re: [openstack-dev] [horizon] REST and Django

________________________________



On Fri Nov 28 2014 at 5:58:00 AM Tripp, Travis S <***@hp.com<mailto:***@hp.com>> wrote:
Hi Richard,

You are right, we should put this out on the main ML, so copying thread out to there. ML: FYI that this started after some impromptu IRC discussions about a specific patch led into an impromptu google hangout discussion with all the people on the thread below.

Thanks Travis!


As I mentioned in the review[1], Thai and I were mainly discussing the possible performance implications of network hops from client to horizon server and whether or not any aggregation should occur server side. In other words, some views require several APIs to be queried before any data can displayed and it would eliminate some extra network requests from client to server if some of the data was first collected on the server side across service APIs. For example, the launch instance wizard will need to collect data from quite a few APIs before even the first step is displayed (I’ve listed those out in the blueprint [2]).

The flip side to that (as you also pointed out) is that if we keep the API’s fine grained then the wizard will be able to optimize in one place the calls for data as it is needed. For example, the first step may only need half of the API calls. It also could lead to perceived performance increases just due to the wizard making a call for different data independently and displaying it as soon as it can.

Indeed, looking at the current launch wizard code it seems like you wouldn't need to load all that data for the wizard to be displayed, since only some subset of it would be necessary to display any given panel of the wizard.


I tend to lean towards your POV and starting with discrete API calls and letting the client optimize calls. If there are performance problems or other reasons then doing data aggregation on the server side could be considered at that point.

I'm glad to hear it. I'm a fan of optimising when necessary, and not beforehand :)


Of course if anybody is able to do some performance testing between the two approaches then that could affect the direction taken.

I would certainly like to see us take some measurements when performance issues pop up. Optimising without solid metrics is bad idea :)


Richard


[1] https://review.openstack.org/#/c/136676/8/openstack_dashboard/api/rest/urls.py
[2] https://blueprints.launchpad.net/horizon/+spec/launch-instance-redesign

-Travis

From: Richard Jones <***@gmail.com<mailto:***@gmail.com>>
Date: Wednesday, November 26, 2014 at 11:55 PM
To: Travis Tripp <***@hp.com<mailto:***@hp.com>>, Thai Q Tran/Silicon Valley/IBM <***@us.ibm.com<mailto:***@us.ibm.com>>, David Lyle <***@gmail.com<mailto:***@gmail.com>>, Maxime Vidori <***@enovance.com<mailto:***@enovance.com>>, "Wroblewski, Szymon" <***@intel.com<mailto:***@intel.com>>, "Wood, Matthew David (HP Cloud - Horizon)" <***@hp.com<mailto:***@hp.com>>, "Chen, Shaoquan" <***@hp.com<mailto:***@hp.com>>, "Farina, Matt (HP Cloud)" <***@hp.com<mailto:***@hp.com>>, Cindy Lu/Silicon Valley/IBM <***@us.ibm.com<mailto:***@us.ibm.com>>, Justin Pomeroy/Rochester/IBM <***@us.ibm.com<mailto:***@us.ibm.com>>, Neill Cox <***@ingenious.com.au<mailto:***@ingenious.com.au>>
Subject: Re: REST and Django

I'm not sure whether this is the appropriate place to discuss this, or whether I should be posting to the list under [Horizon] but I think we need to have a clear idea of what goes in the REST API and what goes in the client (angular) code.

In my mind, the thinner the REST API the better. Indeed if we can get away with proxying requests through without touching any *client code, that would be great.

Coding additional logic into the REST API means that a developer would need to look in two places, instead of one, to determine what was happening for a particular call. If we keep it thin then the API presented to the client developer is very, very similar to the API presented by the services. Minimum surprise.

Your thoughts?


Richard


On Wed Nov 26 2014 at 2:40:52 PM Richard Jones <***@gmail.com<mailto:***@gmail.com>> wrote:
Thanks for the great summary, Travis.

I've completed the work I pledged this morning, so now the REST API change set has:

- no rest framework dependency
- AJAX scaffolding in openstack_dashboard.api.rest.utils
- code in openstack_dashboard/api/rest/
- renamed the API from "identity" to "keystone" to be consistent
- added a sample of testing, mostly for my own sanity to check things were working

https://review.openstack.org/#/c/136676


Richard

On Wed Nov 26 2014 at 12:18:25 PM Tripp, Travis S <***@hp.com<mailto:***@hp.com>> wrote:
Hello all,

Great discussion on the REST urls today! I think that we are on track to come to a common REST API usage pattern. To provide quick summary:

We all agreed that going to a straight REST pattern rather than through tables was a good idea. We discussed using direct get / post in Django views like what Max originally used[1][2] and Thai also started[3] with the identity table rework or to go with djangorestframework [5] like what Richard was prototyping with[4].

The main things we would use from Django Rest Framework were built in JSON serialization (avoid boilerplate), better exception handling, and some request wrapping. However, we all weren’t sure about the need for a full new framework just for that. At the end of the conversation, we decided that it was a cleaner approach, but Richard would see if he could provide some utility code to do that much for us without requiring the full framework. David voiced that he doesn’t want us building out a whole framework on our own either.

So, Richard will do some investigation during his day today and get back to us. Whatever the case, we’ll get a patch in horizon for the base dependency (framework or Richard’s utilities) that both Thai’s work and the launch instance work is dependent upon. We’ll build REST style API’s using the same pattern. We will likely put the rest api’s in horizon/openstack_dashboard/api/rest/.

[1] https://review.openstack.org/#/c/133178/1/openstack_dashboard/workflow/keypair.py
[2] https://review.openstack.org/#/c/133178/1/openstack_dashboard/workflow/launch.py
[3] https://review.openstack.org/#/c/133767/8/openstack_dashboard/dashboards/identity/users/views.py
[4] https://review.openstack.org/#/c/136676/4/openstack_dashboard/rest_api/identity.py
[5] http://www.django-rest-framework.org/

Thanks,
Travis_______________________________________________
OpenStack-dev mailing list
OpenStack-***@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

_______________________________________________
OpenStack-dev mailing list
OpenStack-***@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Richard Jones
2014-12-02 05:39:24 UTC
Permalink
On Mon Dec 01 2014 at 4:18:42 PM Thai Q Tran <***@us.ibm.com> wrote:

> I agree that keeping the API layer thin would be ideal. I should add that
> having discrete API calls would allow dynamic population of table. However,
> I will make a case where it *might* be necessary to add additional APIs.
> Consider that you want to delete 3 items in a given table.
>
> If you do this on the client side, you would need to perform: n * (1 API
> request + 1 AJAX request)
> If you have some logic on the server side that batch delete actions: n *
> (1 API request) + 1 AJAX request
>
> Consider the following:
> n = 1, client = 2 trips, server = 2 trips
> n = 3, client = 6 trips, server = 4 trips
> n = 10, client = 20 trips, server = 11 trips
> n = 100, client = 200 trips, server 101 trips
>
> As you can see, this does not scale very well.... something to consider...
>
Yep, though in the above cases the client is still going to be hanging,
waiting for those server-backend calls, with no feedback until it's all
done. I would hope that the client-server call overhead is minimal, but I
guess that's probably wishful thinking when in the land of random Internet
users hitting some provider's Horizon :)

So yeah, having mulled it over myself I agree that it's useful to have
batch operations implemented in the POST handler, the most common operation
being DELETE.

Maybe one day we could transition to a batch call with user feedback using
a websocket connection.


Richard

> [image: Inactive hide details for Richard Jones ---11/27/2014 05:38:53
> PM---On Fri Nov 28 2014 at 5:58:00 AM Tripp, Travis S <travis.tr]Richard
> Jones ---11/27/2014 05:38:53 PM---On Fri Nov 28 2014 at 5:58:00 AM Tripp,
> Travis S <***@hp.com> wrote:
>
> From: Richard Jones <***@gmail.com>
> To: "Tripp, Travis S" <***@hp.com>, OpenStack List <
> openstack-***@lists.openstack.org>
> Date: 11/27/2014 05:38 PM
> Subject: Re: [openstack-dev] [horizon] REST and Django
> ------------------------------
>
>
>
>
> On Fri Nov 28 2014 at 5:58:00 AM Tripp, Travis S <****@hp.com*
> <***@hp.com>> wrote:
>
> Hi Richard,
>
> You are right, we should put this out on the main ML, so copying
> thread out to there. ML: FYI that this started after some impromptu IRC
> discussions about a specific patch led into an impromptu google hangout
> discussion with all the people on the thread below.
>
>
> Thanks Travis!
>
>
>
> As I mentioned in the review[1], Thai and I were mainly discussing the
> possible performance implications of network hops from client to horizon
> server and whether or not any aggregation should occur server side. In
> other words, some views require several APIs to be queried before any data
> can displayed and it would eliminate some extra network requests from
> client to server if some of the data was first collected on the server side
> across service APIs. For example, the launch instance wizard will need to
> collect data from quite a few APIs before even the first step is displayed
> (I’ve listed those out in the blueprint [2]).
>
> The flip side to that (as you also pointed out) is that if we keep the
> API’s fine grained then the wizard will be able to optimize in one place
> the calls for data as it is needed. For example, the first step may only
> need half of the API calls. It also could lead to perceived performance
> increases just due to the wizard making a call for different data
> independently and displaying it as soon as it can.
>
>
> Indeed, looking at the current launch wizard code it seems like you
> wouldn't need to load all that data for the wizard to be displayed, since
> only some subset of it would be necessary to display any given panel of the
> wizard.
>
>
>
> I tend to lean towards your POV and starting with discrete API calls
> and letting the client optimize calls. If there are performance problems
> or other reasons then doing data aggregation on the server side could be
> considered at that point.
>
>
> I'm glad to hear it. I'm a fan of optimising when necessary, and not
> beforehand :)
>
>
>
> Of course if anybody is able to do some performance testing between
> the two approaches then that could affect the direction taken.
>
>
> I would certainly like to see us take some measurements when performance
> issues pop up. Optimising without solid metrics is bad idea :)
>
>
> Richard
>
>
>
>
> [1]
> *https://review.openstack.org/#/c/136676/8/openstack_dashboard/api/rest/urls.py*
> <https://review.openstack.org/#/c/136676/8/openstack_dashboard/api/rest/urls.py>
> [2]
> *https://blueprints.launchpad.net/horizon/+spec/launch-instance-redesign*
> <https://blueprints.launchpad.net/horizon/+spec/launch-instance-redesign>
>
> -Travis
>
> *From: *Richard Jones <****@gmail.com*
> <***@gmail.com>>
> * Date: *Wednesday, November 26, 2014 at 11:55 PM
> * To: *Travis Tripp <****@hp.com* <***@hp.com>>, Thai Q
> Tran/Silicon Valley/IBM <****@us.ibm.com* <***@us.ibm.com>>,
> David Lyle <****@gmail.com* <***@gmail.com>>, Maxime Vidori <
> ****@enovance.com* <***@enovance.com>>,
> "Wroblewski, Szymon" <****@intel.com*
> <***@intel.com>>, "Wood, Matthew David (HP Cloud -
> Horizon)" <****@hp.com* <***@hp.com>>, "Chen, Shaoquan" <
> ****@hp.com* <***@hp.com>>, "Farina, Matt (HP Cloud)" <
> ****@hp.com* <***@hp.com>>, Cindy Lu/Silicon
> Valley/IBM <****@us.ibm.com* <***@us.ibm.com>>, Justin
> Pomeroy/Rochester/IBM <****@us.ibm.com* <***@us.ibm.com>>,
> Neill Cox <****@ingenious.com.au* <***@ingenious.com.au>>
> * Subject: *Re: REST and Django
>
> I'm not sure whether this is the appropriate place to discuss this, or
> whether I should be posting to the list under [Horizon] but I think we need
> to have a clear idea of what goes in the REST API and what goes in the
> client (angular) code.
>
> In my mind, the thinner the REST API the better. Indeed if we can get
> away with proxying requests through without touching any *client code, that
> would be great.
>
> Coding additional logic into the REST API means that a developer would
> need to look in two places, instead of one, to determine what was happening
> for a particular call. If we keep it thin then the API presented to the
> client developer is very, very similar to the API presented by the
> services. Minimum surprise.
>
> Your thoughts?
>
>
> Richard
>
>
> On Wed Nov 26 2014 at 2:40:52 PM Richard Jones <
> ****@gmail.com* <***@gmail.com>> wrote:
>
>
> Thanks for the great summary, Travis.
>
> I've completed the work I pledged this morning, so now the REST API
> change set has:
>
> - no rest framework dependency
> - AJAX scaffolding in openstack_dashboard.api.rest.utils
> - code in openstack_dashboard/api/rest/
> - renamed the API from "identity" to "keystone" to be consistent
> - added a sample of testing, mostly for my own sanity to check
> things were working
>
> *https://review.openstack.org/#/c/136676*
> <https://review.openstack.org/#/c/136676>
>
>
> Richard
>
> On Wed Nov 26 2014 at 12:18:25 PM Tripp, Travis S <
> ****@hp.com* <***@hp.com>> wrote:
>
>
> Hello all,
>
> Great discussion on the REST urls today! I think that we are on
> track to come to a common REST API usage pattern. To provide quick summary:
>
> We all agreed that going to a straight REST pattern rather than
> through tables was a good idea. We discussed using direct get / post in
> Django views like what Max originally used[1][2] and Thai also started[3]
> with the identity table rework or to go with djangorestframework [5] like
> what Richard was prototyping with[4].
>
> The main things we would use from Django Rest Framework were
> built in JSON serialization (avoid boilerplate), better exception handling,
> and some request wrapping. However, we all weren’t sure about the need for
> a full new framework just for that. At the end of the conversation, we
> decided that it was a cleaner approach, but Richard would see if he could
> provide some utility code to do that much for us without requiring the full
> framework. David voiced that he doesn’t want us building out a whole
> framework on our own either.
>
> So, Richard will do some investigation during his day today and
> get back to us. Whatever the case, we’ll get a patch in horizon for the
> base dependency (framework or Richard’s utilities) that both Thai’s work
> and the launch instance work is dependent upon. We’ll build REST style
> API’s using the same pattern. We will likely put the rest api’s in
> horizon/openstack_dashboard/api/rest/.
>
> [1]
> *https://review.openstack.org/#/c/133178/1/openstack_dashboard/workflow/keypair.py*
> <https://review.openstack.org/#/c/133178/1/openstack_dashboard/workflow/keypair.py>
> [2]
> *https://review.openstack.org/#/c/133178/1/openstack_dashboard/workflow/launch.py*
> <https://review.openstack.org/#/c/133178/1/openstack_dashboard/workflow/launch.py>
> [3]
> *https://review.openstack.org/#/c/133767/8/openstack_dashboard/dashboards/identity/users/views.py*
> <https://review.openstack.org/#/c/133767/8/openstack_dashboard/dashboards/identity/users/views.py>
> [4]
> *https://review.openstack.org/#/c/136676/4/openstack_dashboard/rest_api/identity.py*
> <https://review.openstack.org/#/c/136676/4/openstack_dashboard/rest_api/identity.py>
> [5] *http://www.django-rest-framework.org/*
> <http://www.django-rest-framework.org/>
>
> Thanks,
>
>
> Travis_______________________________________________
> OpenStack-dev mailing list
> OpenStack-***@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-***@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
Adam Young
2014-12-02 14:45:30 UTC
Permalink
On 12/02/2014 12:39 AM, Richard Jones wrote:
> On Mon Dec 01 2014 at 4:18:42 PM Thai Q Tran <***@us.ibm.com
> <mailto:***@us.ibm.com>> wrote:
>
> I agree that keeping the API layer thin would be ideal. I should
> add that having discrete API calls would allow dynamic population
> of table. However, I will make a case where it */might/* be
> necessary to add additional APIs. Consider that you want to delete
> 3 items in a given table.
>
> If you do this on the client side, you would need to perform: n *
> (1 API request + 1 AJAX request)
> If you have some logic on the server side that batch delete
> actions: n * (1 API request) + 1 AJAX request
>
> Consider the following:
> n = 1, client = 2 trips, server = 2 trips
> n = 3, client = 6 trips, server = 4 trips
> n = 10, client = 20 trips, server = 11 trips
> n = 100, client = 200 trips, server 101 trips
>
> As you can see, this does not scale very well.... something to
> consider...
>
This is not something Horizon can fix. Horizon can make matters worse,
but cannot make things better.

If you want to delete 3 users, Horizon still needs to make 3 distinct
calls to Keystone.

To fix this, we need either batch calls or a standard way to do
multiples of the same operation.

The unified API effort it the right place to drive this.






> Yep, though in the above cases the client is still going to be
> hanging, waiting for those server-backend calls, with no feedback
> until it's all done. I would hope that the client-server call overhead
> is minimal, but I guess that's probably wishful thinking when in the
> land of random Internet users hitting some provider's Horizon :)
>
> So yeah, having mulled it over myself I agree that it's useful to have
> batch operations implemented in the POST handler, the most common
> operation being DELETE.
>
> Maybe one day we could transition to a batch call with user feedback
> using a websocket connection.
>
>
> Richard
>
> Inactive hide details for Richard Jones ---11/27/2014 05:38:53
> PM---On Fri Nov 28 2014 at 5:58:00 AM Tripp, Travis S
> <travis.trRichard Jones ---11/27/2014 05:38:53 PM---On Fri Nov 28
> 2014 at 5:58:00 AM Tripp, Travis S <***@hp.com
> <mailto:***@hp.com>> wrote:
>
> From: Richard Jones <***@gmail.com
> <mailto:***@gmail.com>>
> To: "Tripp, Travis S" <***@hp.com
> <mailto:***@hp.com>>, OpenStack List
> <openstack-***@lists.openstack.org
> <mailto:openstack-***@lists.openstack.org>>
> Date: 11/27/2014 05:38 PM
> Subject: Re: [openstack-dev] [horizon] REST and Django
>
> ------------------------------------------------------------------------
>
>
>
>
> On Fri Nov 28 2014 at 5:58:00 AM Tripp, Travis S
> <***@hp.com_ <mailto:***@hp.com>> wrote:
>
> Hi Richard,
>
> You are right, we should put this out on the main ML, so
> copying thread out to there. ML: FYI that this started after
> some impromptu IRC discussions about a specific patch led into
> an impromptu google hangout discussion with all the people on
> the thread below.
>
>
> Thanks Travis!
>
> As I mentioned in the review[1], Thai and I were mainly
> discussing the possible performance implications of network
> hops from client to horizon server and whether or not any
> aggregation should occur server side. In other words, some
> views require several APIs to be queried before any data can
> displayed and it would eliminate some extra network requests
> from client to server if some of the data was first collected
> on the server side across service APIs. For example, the
> launch instance wizard will need to collect data from quite a
> few APIs before even the first step is displayed (I’ve listed
> those out in the blueprint [2]).
>
> The flip side to that (as you also pointed out) is that if we
> keep the API’s fine grained then the wizard will be able to
> optimize in one place the calls for data as it is needed. For
> example, the first step may only need half of the API calls.
> It also could lead to perceived performance increases just due
> to the wizard making a call for different data independently
> and displaying it as soon as it can.
>
>
> Indeed, looking at the current launch wizard code it seems like
> you wouldn't need to load all that data for the wizard to be
> displayed, since only some subset of it would be necessary to
> display any given panel of the wizard.
>
> I tend to lean towards your POV and starting with discrete API
> calls and letting the client optimize calls. If there are
> performance problems or other reasons then doing data
> aggregation on the server side could be considered at that point.
>
>
> I'm glad to hear it. I'm a fan of optimising when necessary, and
> not beforehand :)
>
> Of course if anybody is able to do some performance testing
> between the two approaches then that could affect the
> direction taken.
>
>
> I would certainly like to see us take some measurements when
> performance issues pop up. Optimising without solid metrics is bad
> idea :)
>
>
> Richard
>
>
> [1]
> _https://review.openstack.org/#/c/136676/8/openstack_dashboard/api/rest/urls.py_
> [2]
> _https://blueprints.launchpad.net/horizon/+spec/launch-instance-redesign_
>
> -Travis
>
> *From: *Richard Jones <***@gmail.com_
> <mailto:***@gmail.com>>*
> Date: *Wednesday, November 26, 2014 at 11:55 PM*
> To: *Travis Tripp <***@hp.com_
> <mailto:***@hp.com>>, Thai Q Tran/Silicon Valley/IBM
> <***@us.ibm.com_ <mailto:***@us.ibm.com>>, David Lyle
> <***@gmail.com_ <mailto:***@gmail.com>>, Maxime
> Vidori <***@enovance.com_
> <mailto:***@enovance.com>>, "Wroblewski, Szymon"
> <***@intel.com_
> <mailto:***@intel.com>>, "Wood, Matthew David
> (HP Cloud - Horizon)" <***@hp.com_
> <mailto:***@hp.com>>, "Chen, Shaoquan"
> <***@hp.com_ <mailto:***@hp.com>>, "Farina,
> Matt (HP Cloud)" <***@hp.com_
> <mailto:***@hp.com>>, Cindy Lu/Silicon Valley/IBM
> <***@us.ibm.com_ <mailto:***@us.ibm.com>>, Justin
> Pomeroy/Rochester/IBM <***@us.ibm.com_
> <mailto:***@us.ibm.com>>, Neill Cox
> <***@ingenious.com.au_
> <mailto:***@ingenious.com.au>>*
> Subject: *Re: REST and Django
>
> I'm not sure whether this is the appropriate place to discuss
> this, or whether I should be posting to the list under
> [Horizon] but I think we need to have a clear idea of what
> goes in the REST API and what goes in the client (angular) code.
>
> In my mind, the thinner the REST API the better. Indeed if we
> can get away with proxying requests through without touching
> any *client code, that would be great.
>
> Coding additional logic into the REST API means that a
> developer would need to look in two places, instead of one, to
> determine what was happening for a particular call. If we keep
> it thin then the API presented to the client developer is
> very, very similar to the API presented by the services.
> Minimum surprise.
>
> Your thoughts?
>
>
> Richard
>
>
> On Wed Nov 26 2014 at 2:40:52 PM Richard Jones
> <***@gmail.com_ <mailto:***@gmail.com>> wrote:
>
> Thanks for the great summary, Travis.
>
> I've completed the work I pledged this morning, so now the
> REST API change set has:
>
> - no rest framework dependency
> - AJAX scaffolding in openstack_dashboard.api.rest.utils
> - code in openstack_dashboard/api/rest/
> - renamed the API from "identity" to "keystone" to be
> consistent
> - added a sample of testing, mostly for my own sanity to
> check things were working
>
> _https://review.openstack.org/#/c/136676_
>
>
> Richard
>
> On Wed Nov 26 2014 at 12:18:25 PM Tripp, Travis S
> <***@hp.com_ <mailto:***@hp.com>> wrote:
>
> Hello all,
>
> Great discussion on the REST urls today! I think that
> we are on track to come to a common REST API usage
> pattern. To provide quick summary:
>
> We all agreed that going to a straight REST pattern
> rather than through tables was a good idea. We
> discussed using direct get / post in Django views like
> what Max originally used[1][2] and Thai also
> started[3] with the identity table rework or to go
> with djangorestframework [5] like what Richard was
> prototyping with[4].
>
> The main things we would use from Django Rest
> Framework were built in JSON serialization (avoid
> boilerplate), better exception handling, and some
> request wrapping. However, we all weren’t sure about
> the need for a full new framework just for that. At
> the end of the conversation, we decided that it was a
> cleaner approach, but Richard would see if he could
> provide some utility code to do that much for us
> without requiring the full framework. David voiced
> that he doesn’t want us building out a whole framework
> on our own either.
>
> So, Richard will do some investigation during his day
> today and get back to us. Whatever the case, we’ll
> get a patch in horizon for the base dependency
> (framework or Richard’s utilities) that both Thai’s
> work and the launch instance work is dependent upon.
> We’ll build REST style API’s using the same pattern.
> We will likely put the rest api’s in
> horizon/openstack_dashboard/api/rest/.
>
> [1]
> _https://review.openstack.org/#/c/133178/1/openstack_dashboard/workflow/keypair.py_
> [2]
> _https://review.openstack.org/#/c/133178/1/openstack_dashboard/workflow/launch.py_
> [3]
> _https://review.openstack.org/#/c/133767/8/openstack_dashboard/dashboards/identity/users/views.py_
> [4]
> _https://review.openstack.org/#/c/136676/4/openstack_dashboard/rest_api/identity.py_
> [5] _http://www.django-rest-framework.org/_
>
> Thanks,
>
> Travis_______________________________________________
> OpenStack-dev mailing list
> OpenStack-***@lists.openstack.org
> <mailto:OpenStack-***@lists.openstack.org>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-***@lists.openstack.org
> <mailto:OpenStack-***@lists.openstack.org>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-***@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Tihomir Trifonov
2014-12-09 10:36:08 UTC
Permalink
Sorry for the late reply, just few thoughts on the matter.

IMO the REST middleware should be as thin as possible. And I mean thin in
terms of processing - it should not do pre/post processing of the requests,
but just unpack/pack. So here is an example:

instead of making AJAX calls that contain instructions:

​​
> POST --json --data {"action": "delete", "data": [ {"name":
> "item1"}, {"name": "item2"}, {"name": "item3" ]}


I think a better approach is just to pack/unpack batch commands, and leave
execution to the frontend/backend and not middleware:


​​
> POST --json --data {"
> ​batch
> ":
> ​[
> {​
> "
> ​
> action"
> ​ : "delete"​
> ,
> ​"payload": ​
> {"name": "item1"}
> ​,
> {​
> "
> ​
> action"
> ​ : "delete"​
> ,
> ​
> "payload":
> ​
> {"name": "item
> ​2
> "}
> ​,
> {​
> "
> ​
> action"
> ​ : "delete"​
> ,
> ​
> "payload":
> ​
> {"name": "item
> ​3
> "}
> ​ ] ​
> ​
> ​
> }


​The idea is that the middleware should not know the actual data. It should
ideally just unpack the data:

​​responses = []
>

for cmd in
> ​ ​
> ​
> ​
> request.POST['batch']:​


> ​
> ​​responses
> ​.append(​
> ​
> getattr(controller, cmd['action']
> ​)(**
> cmd['​payload']
> ​)​)
>

> ​return responses​
>


​and the frontend(JS) will just send batches of simple commands, and will
receive a list of responses for each command in the batch. The error
handling will be done in the frontend​(JS) as well.

​

For the more complex example of 'put()' where we have dependent objects:

project = api.keystone.tenant_get(request, id)
> kwargs = self._tenant_kwargs_from_DATA(request.DATA, enabled=None)
> api.keystone.tenant_update(request, project, **kwargs)



In practice the project data should be already present in the
frontend(assuming that we already loaded it to render the project
form/view), so

​
​
POST --json --data {"
​batch
":
​[
{​
"
​
action"
​ : "tenant_update"​
,
​"payload": ​
{"project": js_project_object.id, "name": "some name", "prop1": "some
prop", "prop2": "other prop, etc."}
​
​ ] ​
​
​
}​

So in general we don't need to recreate the full state on each REST call,
if we make the Frontent full-featured application. This way - the frontend
will construct the object, will hold the cached value, and will just send
the needed requests as single ones or in batches, will receive the response
from the API backend, and will render the results. The whole processing
logic will be held in the Frontend(JS), while the middleware will just act
as proxy(un/packer). This way we will maintain just the logic in the
frontend, and will not need to duplicate some logic in the middleware.




On Tue, Dec 2, 2014 at 4:45 PM, Adam Young <***@redhat.com> wrote:

> On 12/02/2014 12:39 AM, Richard Jones wrote:
>
> On Mon Dec 01 2014 at 4:18:42 PM Thai Q Tran <***@us.ibm.com> wrote:
>
>> I agree that keeping the API layer thin would be ideal. I should add
>> that having discrete API calls would allow dynamic population of table.
>> However, I will make a case where it *might* be necessary to add
>> additional APIs. Consider that you want to delete 3 items in a given table.
>>
>> If you do this on the client side, you would need to perform: n * (1 API
>> request + 1 AJAX request)
>> If you have some logic on the server side that batch delete actions: n *
>> (1 API request) + 1 AJAX request
>>
>> Consider the following:
>> n = 1, client = 2 trips, server = 2 trips
>> n = 3, client = 6 trips, server = 4 trips
>> n = 10, client = 20 trips, server = 11 trips
>> n = 100, client = 200 trips, server 101 trips
>>
>> As you can see, this does not scale very well.... something to consider...
>>
> This is not something Horizon can fix. Horizon can make matters worse,
> but cannot make things better.
>
> If you want to delete 3 users, Horizon still needs to make 3 distinct
> calls to Keystone.
>
> To fix this, we need either batch calls or a standard way to do multiples
> of the same operation.
>
> The unified API effort it the right place to drive this.
>
>
>
>
>
>
>
> Yep, though in the above cases the client is still going to be hanging,
> waiting for those server-backend calls, with no feedback until it's all
> done. I would hope that the client-server call overhead is minimal, but I
> guess that's probably wishful thinking when in the land of random Internet
> users hitting some provider's Horizon :)
>
> So yeah, having mulled it over myself I agree that it's useful to have
> batch operations implemented in the POST handler, the most common operation
> being DELETE.
>
> Maybe one day we could transition to a batch call with user feedback
> using a websocket connection.
>
>
> Richard
>
>> [image: Inactive hide details for Richard Jones ---11/27/2014 05:38:53
>> PM---On Fri Nov 28 2014 at 5:58:00 AM Tripp, Travis S <travis.tr]Richard
>> Jones ---11/27/2014 05:38:53 PM---On Fri Nov 28 2014 at 5:58:00 AM Tripp,
>> Travis S <***@hp.com> wrote:
>>
>> From: Richard Jones <***@gmail.com>
>> To: "Tripp, Travis S" <***@hp.com>, OpenStack List <
>> openstack-***@lists.openstack.org>
>> Date: 11/27/2014 05:38 PM
>> Subject: Re: [openstack-dev] [horizon] REST and Django
>> ------------------------------
>>
>>
>>
>>
>> On Fri Nov 28 2014 at 5:58:00 AM Tripp, Travis S <****@hp.com*
>> <***@hp.com>> wrote:
>>
>> Hi Richard,
>>
>> You are right, we should put this out on the main ML, so copying
>> thread out to there. ML: FYI that this started after some impromptu IRC
>> discussions about a specific patch led into an impromptu google hangout
>> discussion with all the people on the thread below.
>>
>>
>> Thanks Travis!
>>
>>
>>
>> As I mentioned in the review[1], Thai and I were mainly discussing
>> the possible performance implications of network hops from client to
>> horizon server and whether or not any aggregation should occur server side.
>> In other words, some views require several APIs to be queried before any
>> data can displayed and it would eliminate some extra network requests from
>> client to server if some of the data was first collected on the server side
>> across service APIs. For example, the launch instance wizard will need to
>> collect data from quite a few APIs before even the first step is displayed
>> (I’ve listed those out in the blueprint [2]).
>>
>> The flip side to that (as you also pointed out) is that if we keep
>> the API’s fine grained then the wizard will be able to optimize in one
>> place the calls for data as it is needed. For example, the first step may
>> only need half of the API calls. It also could lead to perceived
>> performance increases just due to the wizard making a call for different
>> data independently and displaying it as soon as it can.
>>
>>
>> Indeed, looking at the current launch wizard code it seems like you
>> wouldn't need to load all that data for the wizard to be displayed, since
>> only some subset of it would be necessary to display any given panel of the
>> wizard.
>>
>>
>>
>> I tend to lean towards your POV and starting with discrete API calls
>> and letting the client optimize calls. If there are performance problems
>> or other reasons then doing data aggregation on the server side could be
>> considered at that point.
>>
>>
>> I'm glad to hear it. I'm a fan of optimising when necessary, and not
>> beforehand :)
>>
>>
>>
>> Of course if anybody is able to do some performance testing between
>> the two approaches then that could affect the direction taken.
>>
>>
>> I would certainly like to see us take some measurements when performance
>> issues pop up. Optimising without solid metrics is bad idea :)
>>
>>
>> Richard
>>
>>
>>
>> [1]
>> *https://review.openstack.org/#/c/136676/8/openstack_dashboard/api/rest/urls.py*
>> <https://review.openstack.org/#/c/136676/8/openstack_dashboard/api/rest/urls.py>
>> [2]
>> *https://blueprints.launchpad.net/horizon/+spec/launch-instance-redesign*
>> <https://blueprints.launchpad.net/horizon/+spec/launch-instance-redesign>
>>
>> -Travis
>>
>> *From: *Richard Jones <****@gmail.com*
>> <***@gmail.com>>
>> * Date: *Wednesday, November 26, 2014 at 11:55 PM
>> * To: *Travis Tripp <****@hp.com* <***@hp.com>>, Thai
>> Q Tran/Silicon Valley/IBM <****@us.ibm.com* <***@us.ibm.com>>,
>> David Lyle <****@gmail.com* <***@gmail.com>>, Maxime Vidori <
>> ****@enovance.com* <***@enovance.com>>,
>> "Wroblewski, Szymon" <****@intel.com*
>> <***@intel.com>>, "Wood, Matthew David (HP Cloud -
>> Horizon)" <****@hp.com* <***@hp.com>>, "Chen, Shaoquan" <
>> ****@hp.com* <***@hp.com>>, "Farina, Matt (HP Cloud)" <
>> ****@hp.com* <***@hp.com>>, Cindy Lu/Silicon
>> Valley/IBM <****@us.ibm.com* <***@us.ibm.com>>, Justin
>> Pomeroy/Rochester/IBM <****@us.ibm.com* <***@us.ibm.com>>,
>> Neill Cox <****@ingenious.com.au* <***@ingenious.com.au>>
>> * Subject: *Re: REST and Django
>>
>> I'm not sure whether this is the appropriate place to discuss this,
>> or whether I should be posting to the list under [Horizon] but I think we
>> need to have a clear idea of what goes in the REST API and what goes in the
>> client (angular) code.
>>
>> In my mind, the thinner the REST API the better. Indeed if we can get
>> away with proxying requests through without touching any *client code, that
>> would be great.
>>
>> Coding additional logic into the REST API means that a developer
>> would need to look in two places, instead of one, to determine what was
>> happening for a particular call. If we keep it thin then the API presented
>> to the client developer is very, very similar to the API presented by the
>> services. Minimum surprise.
>>
>> Your thoughts?
>>
>>
>> Richard
>>
>>
>> On Wed Nov 26 2014 at 2:40:52 PM Richard Jones <
>> ****@gmail.com* <***@gmail.com>> wrote:
>>
>>
>> Thanks for the great summary, Travis.
>>
>> I've completed the work I pledged this morning, so now the REST
>> API change set has:
>>
>> - no rest framework dependency
>> - AJAX scaffolding in openstack_dashboard.api.rest.utils
>> - code in openstack_dashboard/api/rest/
>> - renamed the API from "identity" to "keystone" to be consistent
>> - added a sample of testing, mostly for my own sanity to check
>> things were working
>>
>> *https://review.openstack.org/#/c/136676*
>> <https://review.openstack.org/#/c/136676>
>>
>>
>> Richard
>>
>> On Wed Nov 26 2014 at 12:18:25 PM Tripp, Travis S <
>> ****@hp.com* <***@hp.com>> wrote:
>>
>>
>> Hello all,
>>
>> Great discussion on the REST urls today! I think that we are on
>> track to come to a common REST API usage pattern. To provide quick summary:
>>
>> We all agreed that going to a straight REST pattern rather than
>> through tables was a good idea. We discussed using direct get / post in
>> Django views like what Max originally used[1][2] and Thai also started[3]
>> with the identity table rework or to go with djangorestframework [5] like
>> what Richard was prototyping with[4].
>>
>> The main things we would use from Django Rest Framework were
>> built in JSON serialization (avoid boilerplate), better exception handling,
>> and some request wrapping. However, we all weren’t sure about the need for
>> a full new framework just for that. At the end of the conversation, we
>> decided that it was a cleaner approach, but Richard would see if he could
>> provide some utility code to do that much for us without requiring the full
>> framework. David voiced that he doesn’t want us building out a whole
>> framework on our own either.
>>
>> So, Richard will do some investigation during his day today and
>> get back to us. Whatever the case, we’ll get a patch in horizon for the
>> base dependency (framework or Richard’s utilities) that both Thai’s work
>> and the launch instance work is dependent upon. We’ll build REST style
>> API’s using the same pattern. We will likely put the rest api’s in
>> horizon/openstack_dashboard/api/rest/.
>>
>> [1]
>> *https://review.openstack.org/#/c/133178/1/openstack_dashboard/workflow/keypair.py*
>> <https://review.openstack.org/#/c/133178/1/openstack_dashboard/workflow/keypair.py>
>> [2]
>> *https://review.openstack.org/#/c/133178/1/openstack_dashboard/workflow/launch.py*
>> <https://review.openstack.org/#/c/133178/1/openstack_dashboard/workflow/launch.py>
>> [3]
>> *https://review.openstack.org/#/c/133767/8/openstack_dashboard/dashboards/identity/users/views.py*
>> <https://review.openstack.org/#/c/133767/8/openstack_dashboard/dashboards/identity/users/views.py>
>> [4]
>> *https://review.openstack.org/#/c/136676/4/openstack_dashboard/rest_api/identity.py*
>> <https://review.openstack.org/#/c/136676/4/openstack_dashboard/rest_api/identity.py>
>> [5] *http://www.django-rest-framework.org/*
>> <http://www.django-rest-framework.org/>
>>
>> Thanks,
>>
>>
>> Travis_______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-***@lists.openstack.org
>>
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-***@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
> _______________________________________________
> OpenStack-dev mailing listOpenStack-***@lists.openstack.orghttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-***@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


--
Regards,
Tihomir Trifonov
Richard Jones
2014-12-10 09:56:06 UTC
Permalink
Sorry I didn't respond to this earlier today, I had intended to.

What you're describing isn't REST, and the principles of REST are what have
been guiding the design of the new API so far. I see a lot of value in
using REST approaches, mostly around clarity of the interface.

While the idea of a very thin proxy seemed like a great idea at one point,
my conversations at the summit convinced me that there was value in both
using the client interfaces present in the openstack_dashboard/api code
base (since they abstract away many issues in the apis including across
versions) and also value in us being able to clean up (for example, using
"project_id" rather than "project" in the user API we've already
implemented) and extend those interfaces (to allow batched operations).

We want to be careful about what we expose in Horizon to the JS clients
through this API. That necessitates some amount of code in Horizon. About
half of the current API for keysone represents that control (the other half
is docstrings :)


Richard


On Tue Dec 09 2014 at 9:37:47 PM Tihomir Trifonov <***@gmail.com>
wrote:

> Sorry for the late reply, just few thoughts on the matter.
>
> IMO the REST middleware should be as thin as possible. And I mean thin in
> terms of processing - it should not do pre/post processing of the requests,
> but just unpack/pack. So here is an example:
>
> instead of making AJAX calls that contain instructions:
>
> ​​
>> POST --json --data {"action": "delete", "data": [ {"name":
>> "item1"}, {"name": "item2"}, {"name": "item3" ]}
>
>
> I think a better approach is just to pack/unpack batch commands, and leave
> execution to the frontend/backend and not middleware:
>
>
> ​​
>> POST --json --data {"
>> ​batch
>> ":
>> ​[
>> {​
>> "
>> ​
>> action"
>> ​ : "delete"​
>> ,
>> ​"payload": ​
>> {"name": "item1"}
>> ​,
>> {​
>> "
>> ​
>> action"
>> ​ : "delete"​
>> ,
>> ​
>> "payload":
>> ​
>> {"name": "item
>> ​2
>> "}
>> ​,
>> {​
>> "
>> ​
>> action"
>> ​ : "delete"​
>> ,
>> ​
>> "payload":
>> ​
>> {"name": "item
>> ​3
>> "}
>> ​ ] ​
>> ​
>> ​
>> }
>
>
> ​The idea is that the middleware should not know the actual data. It
> should ideally just unpack the data:
>
> ​​responses = []
>>
>
> for cmd in
>> ​ ​
>> ​
>> ​
>> request.POST['batch']:​
>
>
>> ​
>> ​​responses
>> ​.append(​
>> ​
>> getattr(controller, cmd['action']
>> ​)(**
>> cmd['​payload']
>> ​)​)
>>
>
>> ​return responses​
>>
>
>
> ​and the frontend(JS) will just send batches of simple commands, and will
> receive a list of responses for each command in the batch. The error
> handling will be done in the frontend​(JS) as well.
>
> ​
>
> For the more complex example of 'put()' where we have dependent objects:
>
> project = api.keystone.tenant_get(request, id)
>> kwargs = self._tenant_kwargs_from_DATA(request.DATA, enabled=None)
>> api.keystone.tenant_update(request, project, **kwargs)
>
>
>
> In practice the project data should be already present in the
> frontend(assuming that we already loaded it to render the project
> form/view), so
>
> ​
> ​
> POST --json --data {"
> ​batch
> ":
> ​[
> {​
> "
> ​
> action"
> ​ : "tenant_update"​
> ,
> ​"payload": ​
> {"project": js_project_object.id, "name": "some name", "prop1": "some
> prop", "prop2": "other prop, etc."}
> ​
> ​ ] ​
> ​
> ​
> }​
>
> So in general we don't need to recreate the full state on each REST call,
> if we make the Frontent full-featured application. This way - the frontend
> will construct the object, will hold the cached value, and will just send
> the needed requests as single ones or in batches, will receive the response
> from the API backend, and will render the results. The whole processing
> logic will be held in the Frontend(JS), while the middleware will just act
> as proxy(un/packer). This way we will maintain just the logic in the
> frontend, and will not need to duplicate some logic in the middleware.
>
>
>
>
> On Tue, Dec 2, 2014 at 4:45 PM, Adam Young <***@redhat.com> wrote:
>
>> On 12/02/2014 12:39 AM, Richard Jones wrote:
>>
>> On Mon Dec 01 2014 at 4:18:42 PM Thai Q Tran <***@us.ibm.com> wrote:
>>
>>> I agree that keeping the API layer thin would be ideal. I should add
>>> that having discrete API calls would allow dynamic population of table.
>>> However, I will make a case where it *might* be necessary to add
>>> additional APIs. Consider that you want to delete 3 items in a given table.
>>>
>>> If you do this on the client side, you would need to perform: n * (1 API
>>> request + 1 AJAX request)
>>> If you have some logic on the server side that batch delete actions: n *
>>> (1 API request) + 1 AJAX request
>>>
>>> Consider the following:
>>> n = 1, client = 2 trips, server = 2 trips
>>> n = 3, client = 6 trips, server = 4 trips
>>> n = 10, client = 20 trips, server = 11 trips
>>> n = 100, client = 200 trips, server 101 trips
>>>
>>> As you can see, this does not scale very well.... something to
>>> consider...
>>>
>> This is not something Horizon can fix. Horizon can make matters worse,
>> but cannot make things better.
>>
>> If you want to delete 3 users, Horizon still needs to make 3 distinct
>> calls to Keystone.
>>
>> To fix this, we need either batch calls or a standard way to do multiples
>> of the same operation.
>>
>> The unified API effort it the right place to drive this.
>>
>>
>>
>>
>>
>>
>>
>> Yep, though in the above cases the client is still going to be hanging,
>> waiting for those server-backend calls, with no feedback until it's all
>> done. I would hope that the client-server call overhead is minimal, but I
>> guess that's probably wishful thinking when in the land of random Internet
>> users hitting some provider's Horizon :)
>>
>> So yeah, having mulled it over myself I agree that it's useful to have
>> batch operations implemented in the POST handler, the most common operation
>> being DELETE.
>>
>> Maybe one day we could transition to a batch call with user feedback
>> using a websocket connection.
>>
>>
>> Richard
>>
>>> Richard Jones ---11/27/2014 05:38:53 PM---On Fri Nov 28 2014 at
>>> 5:58:00 AM Tripp, Travis S <***@hp.com> wrote:
>>>
>>> From: Richard Jones <***@gmail.com>
>>> To: "Tripp, Travis S" <***@hp.com>, OpenStack List <
>>> openstack-***@lists.openstack.org>
>>> Date: 11/27/2014 05:38 PM
>>> Subject: Re: [openstack-dev] [horizon] REST and Django
>>>
>>> ------------------------------
>>>
>>>
>>>
>>>
>>> On Fri Nov 28 2014 at 5:58:00 AM Tripp, Travis S <****@hp.com*
>>> <***@hp.com>> wrote:
>>>
>>> Hi Richard,
>>>
>>> You are right, we should put this out on the main ML, so copying
>>> thread out to there. ML: FYI that this started after some impromptu IRC
>>> discussions about a specific patch led into an impromptu google hangout
>>> discussion with all the people on the thread below.
>>>
>>>
>>> Thanks Travis!
>>>
>>>
>>>
>>> As I mentioned in the review[1], Thai and I were mainly discussing
>>> the possible performance implications of network hops from client to
>>> horizon server and whether or not any aggregation should occur server side.
>>> In other words, some views require several APIs to be queried before any
>>> data can displayed and it would eliminate some extra network requests from
>>> client to server if some of the data was first collected on the server side
>>> across service APIs. For example, the launch instance wizard will need to
>>> collect data from quite a few APIs before even the first step is displayed
>>> (I’ve listed those out in the blueprint [2]).
>>>
>>> The flip side to that (as you also pointed out) is that if we keep
>>> the API’s fine grained then the wizard will be able to optimize in one
>>> place the calls for data as it is needed. For example, the first step may
>>> only need half of the API calls. It also could lead to perceived
>>> performance increases just due to the wizard making a call for different
>>> data independently and displaying it as soon as it can.
>>>
>>>
>>> Indeed, looking at the current launch wizard code it seems like you
>>> wouldn't need to load all that data for the wizard to be displayed, since
>>> only some subset of it would be necessary to display any given panel of the
>>> wizard.
>>>
>>>
>>>
>>> I tend to lean towards your POV and starting with discrete API calls
>>> and letting the client optimize calls. If there are performance problems
>>> or other reasons then doing data aggregation on the server side could be
>>> considered at that point.
>>>
>>>
>>> I'm glad to hear it. I'm a fan of optimising when necessary, and not
>>> beforehand :)
>>>
>>>
>>>
>>> Of course if anybody is able to do some performance testing between
>>> the two approaches then that could affect the direction taken.
>>>
>>>
>>> I would certainly like to see us take some measurements when performance
>>> issues pop up. Optimising without solid metrics is bad idea :)
>>>
>>>
>>> Richard
>>>
>>>
>>>
>>> [1]
>>> *https://review.openstack.org/#/c/136676/8/openstack_dashboard/api/rest/urls.py*
>>> <https://review.openstack.org/#/c/136676/8/openstack_dashboard/api/rest/urls.py>
>>> [2]
>>> *https://blueprints.launchpad.net/horizon/+spec/launch-instance-redesign*
>>> <https://blueprints.launchpad.net/horizon/+spec/launch-instance-redesign>
>>>
>>> -Travis
>>>
>>> *From: *Richard Jones <****@gmail.com*
>>> <***@gmail.com>>
>>> * Date: *Wednesday, November 26, 2014 at 11:55 PM
>>> * To: *Travis Tripp <****@hp.com* <***@hp.com>>, Thai
>>> Q Tran/Silicon Valley/IBM <****@us.ibm.com* <***@us.ibm.com>>,
>>> David Lyle <****@gmail.com* <***@gmail.com>>, Maxime Vidori <
>>> ****@enovance.com* <***@enovance.com>>,
>>> "Wroblewski, Szymon" <****@intel.com*
>>> <***@intel.com>>, "Wood, Matthew David (HP Cloud -
>>> Horizon)" <****@hp.com* <***@hp.com>>, "Chen, Shaoquan" <
>>> ****@hp.com* <***@hp.com>>, "Farina, Matt (HP Cloud)" <
>>> ****@hp.com* <***@hp.com>>, Cindy Lu/Silicon
>>> Valley/IBM <****@us.ibm.com* <***@us.ibm.com>>, Justin
>>> Pomeroy/Rochester/IBM <****@us.ibm.com* <***@us.ibm.com>>,
>>> Neill Cox <****@ingenious.com.au* <***@ingenious.com.au>>
>>> * Subject: *Re: REST and Django
>>>
>>> I'm not sure whether this is the appropriate place to discuss this,
>>> or whether I should be posting to the list under [Horizon] but I think we
>>> need to have a clear idea of what goes in the REST API and what goes in the
>>> client (angular) code.
>>>
>>> In my mind, the thinner the REST API the better. Indeed if we can
>>> get away with proxying requests through without touching any *client code,
>>> that would be great.
>>>
>>> Coding additional logic into the REST API means that a developer
>>> would need to look in two places, instead of one, to determine what was
>>> happening for a particular call. If we keep it thin then the API presented
>>> to the client developer is very, very similar to the API presented by the
>>> services. Minimum surprise.
>>>
>>> Your thoughts?
>>>
>>>
>>> Richard
>>>
>>>
>>> On Wed Nov 26 2014 at 2:40:52 PM Richard Jones <
>>> ****@gmail.com* <***@gmail.com>> wrote:
>>>
>>>
>>> Thanks for the great summary, Travis.
>>>
>>> I've completed the work I pledged this morning, so now the REST
>>> API change set has:
>>>
>>> - no rest framework dependency
>>> - AJAX scaffolding in openstack_dashboard.api.rest.utils
>>> - code in openstack_dashboard/api/rest/
>>> - renamed the API from "identity" to "keystone" to be consistent
>>> - added a sample of testing, mostly for my own sanity to check
>>> things were working
>>>
>>> *https://review.openstack.org/#/c/136676*
>>> <https://review.openstack.org/#/c/136676>
>>>
>>>
>>> Richard
>>>
>>> On Wed Nov 26 2014 at 12:18:25 PM Tripp, Travis S <
>>> ****@hp.com* <***@hp.com>> wrote:
>>>
>>>
>>> Hello all,
>>>
>>> Great discussion on the REST urls today! I think that we are
>>> on track to come to a common REST API usage pattern. To provide quick
>>> summary:
>>>
>>> We all agreed that going to a straight REST pattern rather
>>> than through tables was a good idea. We discussed using direct get / post
>>> in Django views like what Max originally used[1][2] and Thai also
>>> started[3] with the identity table rework or to go with djangorestframework
>>> [5] like what Richard was prototyping with[4].
>>>
>>> The main things we would use from Django Rest Framework were
>>> built in JSON serialization (avoid boilerplate), better exception handling,
>>> and some request wrapping. However, we all weren’t sure about the need for
>>> a full new framework just for that. At the end of the conversation, we
>>> decided that it was a cleaner approach, but Richard would see if he could
>>> provide some utility code to do that much for us without requiring the full
>>> framework. David voiced that he doesn’t want us building out a whole
>>> framework on our own either.
>>>
>>> So, Richard will do some investigation during his day today
>>> and get back to us. Whatever the case, we’ll get a patch in horizon for
>>> the base dependency (framework or Richard’s utilities) that both Thai’s
>>> work and the launch instance work is dependent upon. We’ll build REST
>>> style API’s using the same pattern. We will likely put the rest api’s in
>>> horizon/openstack_dashboard/api/rest/.
>>>
>>> [1]
>>> *https://review.openstack.org/#/c/133178/1/openstack_dashboard/workflow/keypair.py*
>>> <https://review.openstack.org/#/c/133178/1/openstack_dashboard/workflow/keypair.py>
>>> [2]
>>> *https://review.openstack.org/#/c/133178/1/openstack_dashboard/workflow/launch.py*
>>> <https://review.openstack.org/#/c/133178/1/openstack_dashboard/workflow/launch.py>
>>> [3]
>>> *https://review.openstack.org/#/c/133767/8/openstack_dashboard/dashboards/identity/users/views.py*
>>> <https://review.openstack.org/#/c/133767/8/openstack_dashboard/dashboards/identity/users/views.py>
>>> [4]
>>> *https://review.openstack.org/#/c/136676/4/openstack_dashboard/rest_api/identity.py*
>>> <https://review.openstack.org/#/c/136676/4/openstack_dashboard/rest_api/identity.py>
>>> [5] *http://www.django-rest-framework.org/*
>>> <http://www.django-rest-framework.org/>
>>>
>>> Thanks,
>>>
>>>
>>> Travis_______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-***@lists.openstack.org
>>>
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-***@lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
>>
>> _______________________________________________
>> OpenStack-dev mailing listOpenStack-***@lists.openstack.orghttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-***@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
>
> --
> Regards,
> Tihomir Trifonov
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-***@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
Tihomir Trifonov
2014-12-10 11:02:27 UTC
Permalink
Richard, thanks for the reply,


I agree that the given example is not a real REST. But we already have the
REST API - that's Keystone, Nova, Cinder, Glance, Neutron etc, APIs. So
what we plan to do here? To add a new REST layer to communicate with other
REST API? Do we really need Frontend-REST-REST architecture ? My opinion is
that we don't need another REST layer as we currently are trying to go away
from the Django layer, which is the same - another processing layer.
Although we call it REST proxy or whatever - it doesn't need to be a real
REST, but just an aggregation proxy that combines and forwards some
requests with adding minimal processing overhead. What makes sense for me
is to keep the authentication in this layer as it is now - push a cookie to
the frontend, but the REST layer will extract the auth tokens from the
session storage and prepare the auth context for the REST API request to OS
services. This way we will not expose the tokens to the JS frontend, and
will have strict control over the authentication. The frontend will just
send data requests, they will be wrapped with auth context and forwarded.

Regarding the existing issues with versions in the API - for me the
existing approach is wrong. All these fixes were made as workarounds. What
should have been done is to create abstractions for each version and to use
a separate class for each version. This was partially done for the
keystoneclient in api/keystone.py, but not for the forms/views, where we
still have if-else for versions. What I suggest here is to have
different(concrete) views/forms for each version, and to use them according
the context. If the Keystone backend is v2.0 - then in the Frontend use
keystone2() object, otherwise use keystone3() object. This of course needs
some more coding, but is much cleaner in terms of customization and
testing. For me the current hacks with 'if keystone.version == 3.0' are
wrong at many levels. And this can be solved now. *The problem till now was
that we had one frontend that had to be backed by different versions of
backend components*. *Now we can have different frontends that map to
specific backend*. That's how I understand the power of Angular with it's
views and directives. That's where I see the real benefit of using
full-featured frontend. Also imagine how easy will be then to deprecate a
component version, compared to what we need to do now for the same.

Otherwise we just rewrite the current Django middleware with another
DjangoRest middleware and don't change anything, we don't fix the problems.
We just move them to another place.

I still think that in Paris we talked about a new generation of the
Dashboard, a different approach on building the frontend for OpenStack.
What I've heard there from users/operators of Horizon was that it was
extremely hard to add customizations and new features to the Dashboard, as
all these needed to go through upstream changes and to wait until next
release cycle to get them. Do we still want to address these concerns and
how? Please, correct me if I got things wrong.


On Wed, Dec 10, 2014 at 11:56 AM, Richard Jones <***@gmail.com>
wrote:

> Sorry I didn't respond to this earlier today, I had intended to.
>
> What you're describing isn't REST, and the principles of REST are what
> have been guiding the design of the new API so far. I see a lot of value in
> using REST approaches, mostly around clarity of the interface.
>
> While the idea of a very thin proxy seemed like a great idea at one point,
> my conversations at the summit convinced me that there was value in both
> using the client interfaces present in the openstack_dashboard/api code
> base (since they abstract away many issues in the apis including across
> versions) and also value in us being able to clean up (for example, using
> "project_id" rather than "project" in the user API we've already
> implemented) and extend those interfaces (to allow batched operations).
>
> We want to be careful about what we expose in Horizon to the JS clients
> through this API. That necessitates some amount of code in Horizon. About
> half of the current API for keysone represents that control (the other half
> is docstrings :)
>
>
> Richard
>
>
> On Tue Dec 09 2014 at 9:37:47 PM Tihomir Trifonov <***@gmail.com>
> wrote:
>
>> Sorry for the late reply, just few thoughts on the matter.
>>
>> IMO the REST middleware should be as thin as possible. And I mean thin in
>> terms of processing - it should not do pre/post processing of the requests,
>> but just unpack/pack. So here is an example:
>>
>> instead of making AJAX calls that contain instructions:
>>
>> ​​
>>> POST --json --data {"action": "delete", "data": [ {"name":
>>> "item1"}, {"name": "item2"}, {"name": "item3" ]}
>>
>>
>> I think a better approach is just to pack/unpack batch commands, and
>> leave execution to the frontend/backend and not middleware:
>>
>>
>> ​​
>>> POST --json --data {"
>>> ​batch
>>> ":
>>> ​[
>>> {​
>>> "
>>> ​
>>> action"
>>> ​ : "delete"​
>>> ,
>>> ​"payload": ​
>>> {"name": "item1"}
>>> ​,
>>> {​
>>> "
>>> ​
>>> action"
>>> ​ : "delete"​
>>> ,
>>> ​
>>> "payload":
>>> ​
>>> {"name": "item
>>> ​2
>>> "}
>>> ​,
>>> {​
>>> "
>>> ​
>>> action"
>>> ​ : "delete"​
>>> ,
>>> ​
>>> "payload":
>>> ​
>>> {"name": "item
>>> ​3
>>> "}
>>> ​ ] ​
>>> ​
>>> ​
>>> }
>>
>>
>> ​The idea is that the middleware should not know the actual data. It
>> should ideally just unpack the data:
>>
>> ​​responses = []
>>>
>>
>> for cmd in
>>> ​ ​
>>> ​
>>> ​
>>> request.POST['batch']:​
>>
>>
>>> ​
>>> ​​responses
>>> ​.append(​
>>> ​
>>> getattr(controller, cmd['action']
>>> ​)(**
>>> cmd['​payload']
>>> ​)​)
>>>
>>
>>> ​return responses​
>>>
>>
>>
>> ​and the frontend(JS) will just send batches of simple commands, and will
>> receive a list of responses for each command in the batch. The error
>> handling will be done in the frontend​(JS) as well.
>>
>> ​
>>
>> For the more complex example of 'put()' where we have dependent objects:
>>
>> project = api.keystone.tenant_get(request, id)
>>> kwargs = self._tenant_kwargs_from_DATA(request.DATA, enabled=None)
>>> api.keystone.tenant_update(request, project, **kwargs)
>>
>>
>>
>> In practice the project data should be already present in the
>> frontend(assuming that we already loaded it to render the project
>> form/view), so
>>
>> ​
>> ​
>> POST --json --data {"
>> ​batch
>> ":
>> ​[
>> {​
>> "
>> ​
>> action"
>> ​ : "tenant_update"​
>> ,
>> ​"payload": ​
>> {"project": js_project_object.id, "name": "some name", "prop1": "some
>> prop", "prop2": "other prop, etc."}
>> ​
>> ​ ] ​
>> ​
>> ​
>> }​
>>
>> So in general we don't need to recreate the full state on each REST call,
>> if we make the Frontent full-featured application. This way - the frontend
>> will construct the object, will hold the cached value, and will just send
>> the needed requests as single ones or in batches, will receive the response
>> from the API backend, and will render the results. The whole processing
>> logic will be held in the Frontend(JS), while the middleware will just act
>> as proxy(un/packer). This way we will maintain just the logic in the
>> frontend, and will not need to duplicate some logic in the middleware.
>>
>>
>>
>>
>> On Tue, Dec 2, 2014 at 4:45 PM, Adam Young <***@redhat.com> wrote:
>>
>>> On 12/02/2014 12:39 AM, Richard Jones wrote:
>>>
>>> On Mon Dec 01 2014 at 4:18:42 PM Thai Q Tran <***@us.ibm.com> wrote:
>>>
>>>> I agree that keeping the API layer thin would be ideal. I should add
>>>> that having discrete API calls would allow dynamic population of table.
>>>> However, I will make a case where it *might* be necessary to add
>>>> additional APIs. Consider that you want to delete 3 items in a given table.
>>>>
>>>> If you do this on the client side, you would need to perform: n * (1
>>>> API request + 1 AJAX request)
>>>> If you have some logic on the server side that batch delete actions: n
>>>> * (1 API request) + 1 AJAX request
>>>>
>>>> Consider the following:
>>>> n = 1, client = 2 trips, server = 2 trips
>>>> n = 3, client = 6 trips, server = 4 trips
>>>> n = 10, client = 20 trips, server = 11 trips
>>>> n = 100, client = 200 trips, server 101 trips
>>>>
>>>> As you can see, this does not scale very well.... something to
>>>> consider...
>>>>
>>> This is not something Horizon can fix. Horizon can make matters
>>> worse, but cannot make things better.
>>>
>>> If you want to delete 3 users, Horizon still needs to make 3 distinct
>>> calls to Keystone.
>>>
>>> To fix this, we need either batch calls or a standard way to do
>>> multiples of the same operation.
>>>
>>> The unified API effort it the right place to drive this.
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> Yep, though in the above cases the client is still going to be
>>> hanging, waiting for those server-backend calls, with no feedback until
>>> it's all done. I would hope that the client-server call overhead is
>>> minimal, but I guess that's probably wishful thinking when in the land of
>>> random Internet users hitting some provider's Horizon :)
>>>
>>> So yeah, having mulled it over myself I agree that it's useful to have
>>> batch operations implemented in the POST handler, the most common operation
>>> being DELETE.
>>>
>>> Maybe one day we could transition to a batch call with user feedback
>>> using a websocket connection.
>>>
>>>
>>> Richard
>>>
>>>> Richard Jones ---11/27/2014 05:38:53 PM---On Fri Nov 28 2014 at
>>>> 5:58:00 AM Tripp, Travis S <***@hp.com> wrote:
>>>>
>>>> From: Richard Jones <***@gmail.com>
>>>> To: "Tripp, Travis S" <***@hp.com>, OpenStack List <
>>>> openstack-***@lists.openstack.org>
>>>> Date: 11/27/2014 05:38 PM
>>>> Subject: Re: [openstack-dev] [horizon] REST and Django
>>>>
>>>> ------------------------------
>>>>
>>>>
>>>>
>>>>
>>>> On Fri Nov 28 2014 at 5:58:00 AM Tripp, Travis S <****@hp.com*
>>>> <***@hp.com>> wrote:
>>>>
>>>> Hi Richard,
>>>>
>>>> You are right, we should put this out on the main ML, so copying
>>>> thread out to there. ML: FYI that this started after some impromptu IRC
>>>> discussions about a specific patch led into an impromptu google hangout
>>>> discussion with all the people on the thread below.
>>>>
>>>>
>>>> Thanks Travis!
>>>>
>>>>
>>>>
>>>> As I mentioned in the review[1], Thai and I were mainly discussing
>>>> the possible performance implications of network hops from client to
>>>> horizon server and whether or not any aggregation should occur server side.
>>>> In other words, some views require several APIs to be queried before any
>>>> data can displayed and it would eliminate some extra network requests from
>>>> client to server if some of the data was first collected on the server side
>>>> across service APIs. For example, the launch instance wizard will need to
>>>> collect data from quite a few APIs before even the first step is displayed
>>>> (I’ve listed those out in the blueprint [2]).
>>>>
>>>> The flip side to that (as you also pointed out) is that if we keep
>>>> the API’s fine grained then the wizard will be able to optimize in one
>>>> place the calls for data as it is needed. For example, the first step may
>>>> only need half of the API calls. It also could lead to perceived
>>>> performance increases just due to the wizard making a call for different
>>>> data independently and displaying it as soon as it can.
>>>>
>>>>
>>>> Indeed, looking at the current launch wizard code it seems like you
>>>> wouldn't need to load all that data for the wizard to be displayed, since
>>>> only some subset of it would be necessary to display any given panel of the
>>>> wizard.
>>>>
>>>>
>>>>
>>>> I tend to lean towards your POV and starting with discrete API
>>>> calls and letting the client optimize calls. If there are performance
>>>> problems or other reasons then doing data aggregation on the server side
>>>> could be considered at that point.
>>>>
>>>>
>>>> I'm glad to hear it. I'm a fan of optimising when necessary, and not
>>>> beforehand :)
>>>>
>>>>
>>>>
>>>> Of course if anybody is able to do some performance testing between
>>>> the two approaches then that could affect the direction taken.
>>>>
>>>>
>>>> I would certainly like to see us take some measurements when
>>>> performance issues pop up. Optimising without solid metrics is bad idea :)
>>>>
>>>>
>>>> Richard
>>>>
>>>>
>>>>
>>>> [1]
>>>> *https://review.openstack.org/#/c/136676/8/openstack_dashboard/api/rest/urls.py*
>>>> <https://review.openstack.org/#/c/136676/8/openstack_dashboard/api/rest/urls.py>
>>>> [2]
>>>> *https://blueprints.launchpad.net/horizon/+spec/launch-instance-redesign*
>>>> <https://blueprints.launchpad.net/horizon/+spec/launch-instance-redesign>
>>>>
>>>> -Travis
>>>>
>>>> *From: *Richard Jones <****@gmail.com*
>>>> <***@gmail.com>>
>>>> * Date: *Wednesday, November 26, 2014 at 11:55 PM
>>>> * To: *Travis Tripp <****@hp.com* <***@hp.com>>,
>>>> Thai Q Tran/Silicon Valley/IBM <****@us.ibm.com*
>>>> <***@us.ibm.com>>, David Lyle <****@gmail.com*
>>>> <***@gmail.com>>, Maxime Vidori <****@enovance.com*
>>>> <***@enovance.com>>, "Wroblewski, Szymon" <
>>>> ****@intel.com* <***@intel.com>>,
>>>> "Wood, Matthew David (HP Cloud - Horizon)" <****@hp.com*
>>>> <***@hp.com>>, "Chen, Shaoquan" <****@hp.com*
>>>> <***@hp.com>>, "Farina, Matt (HP Cloud)" <
>>>> ****@hp.com* <***@hp.com>>, Cindy Lu/Silicon
>>>> Valley/IBM <****@us.ibm.com* <***@us.ibm.com>>, Justin
>>>> Pomeroy/Rochester/IBM <****@us.ibm.com* <***@us.ibm.com>>,
>>>> Neill Cox <****@ingenious.com.au* <***@ingenious.com.au>
>>>> >
>>>> * Subject: *Re: REST and Django
>>>>
>>>> I'm not sure whether this is the appropriate place to discuss this,
>>>> or whether I should be posting to the list under [Horizon] but I think we
>>>> need to have a clear idea of what goes in the REST API and what goes in the
>>>> client (angular) code.
>>>>
>>>> In my mind, the thinner the REST API the better. Indeed if we can
>>>> get away with proxying requests through without touching any *client code,
>>>> that would be great.
>>>>
>>>> Coding additional logic into the REST API means that a developer
>>>> would need to look in two places, instead of one, to determine what was
>>>> happening for a particular call. If we keep it thin then the API presented
>>>> to the client developer is very, very similar to the API presented by the
>>>> services. Minimum surprise.
>>>>
>>>> Your thoughts?
>>>>
>>>>
>>>> Richard
>>>>
>>>>
>>>> On Wed Nov 26 2014 at 2:40:52 PM Richard Jones <
>>>> ****@gmail.com* <***@gmail.com>> wrote:
>>>>
>>>>
>>>> Thanks for the great summary, Travis.
>>>>
>>>> I've completed the work I pledged this morning, so now the REST
>>>> API change set has:
>>>>
>>>> - no rest framework dependency
>>>> - AJAX scaffolding in openstack_dashboard.api.rest.utils
>>>> - code in openstack_dashboard/api/rest/
>>>> - renamed the API from "identity" to "keystone" to be consistent
>>>> - added a sample of testing, mostly for my own sanity to check
>>>> things were working
>>>>
>>>> *https://review.openstack.org/#/c/136676*
>>>> <https://review.openstack.org/#/c/136676>
>>>>
>>>>
>>>> Richard
>>>>
>>>> On Wed Nov 26 2014 at 12:18:25 PM Tripp, Travis S <
>>>> ****@hp.com* <***@hp.com>> wrote:
>>>>
>>>>
>>>> Hello all,
>>>>
>>>> Great discussion on the REST urls today! I think that we are
>>>> on track to come to a common REST API usage pattern. To provide quick
>>>> summary:
>>>>
>>>> We all agreed that going to a straight REST pattern rather
>>>> than through tables was a good idea. We discussed using direct get / post
>>>> in Django views like what Max originally used[1][2] and Thai also
>>>> started[3] with the identity table rework or to go with djangorestframework
>>>> [5] like what Richard was prototyping with[4].
>>>>
>>>> The main things we would use from Django Rest Framework were
>>>> built in JSON serialization (avoid boilerplate), better exception handling,
>>>> and some request wrapping. However, we all weren’t sure about the need for
>>>> a full new framework just for that. At the end of the conversation, we
>>>> decided that it was a cleaner approach, but Richard would see if he could
>>>> provide some utility code to do that much for us without requiring the full
>>>> framework. David voiced that he doesn’t want us building out a whole
>>>> framework on our own either.
>>>>
>>>> So, Richard will do some investigation during his day today
>>>> and get back to us. Whatever the case, we’ll get a patch in horizon for
>>>> the base dependency (framework or Richard’s utilities) that both Thai’s
>>>> work and the launch instance work is dependent upon. We’ll build REST
>>>> style API’s using the same pattern. We will likely put the rest api’s in
>>>> horizon/openstack_dashboard/api/rest/.
>>>>
>>>> [1]
>>>> *https://review.openstack.org/#/c/133178/1/openstack_dashboard/workflow/keypair.py*
>>>> <https://review.openstack.org/#/c/133178/1/openstack_dashboard/workflow/keypair.py>
>>>> [2]
>>>> *https://review.openstack.org/#/c/133178/1/openstack_dashboard/workflow/launch.py*
>>>> <https://review.openstack.org/#/c/133178/1/openstack_dashboard/workflow/launch.py>
>>>> [3]
>>>> *https://review.openstack.org/#/c/133767/8/openstack_dashboard/dashboards/identity/users/views.py*
>>>> <https://review.openstack.org/#/c/133767/8/openstack_dashboard/dashboards/identity/users/views.py>
>>>> [4]
>>>> *https://review.openstack.org/#/c/136676/4/openstack_dashboard/rest_api/identity.py*
>>>> <https://review.openstack.org/#/c/136676/4/openstack_dashboard/rest_api/identity.py>
>>>> [5] *http://www.django-rest-framework.org/*
>>>> <http://www.django-rest-framework.org/>
>>>>
>>>> Thanks,
>>>>
>>>>
>>>> Travis_______________________________________________
>>>> OpenStack-dev mailing list
>>>> OpenStack-***@lists.openstack.org
>>>>
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>>
>>>> _______________________________________________
>>>> OpenStack-dev mailing list
>>>> OpenStack-***@lists.openstack.org
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>
>>>
>>> _______________________________________________
>>> OpenStack-dev mailing listOpenStack-***@lists.openstack.orghttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>>
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-***@lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>
>>
>> --
>> Regards,
>> Tihomir Trifonov
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-***@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-***@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


--
Regards,
Tihomir Trifonov
Thai Q Tran
2014-12-10 18:39:07 UTC
Permalink
_______________________________________________
OpenStack-dev mailing list
OpenStack-***@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Tihomir Trifonov
2014-12-11 14:53:16 UTC
Permalink
>
> *​​Client just needs to know which URL to hit in order to invoke a certain
> API, and does not need to know the procedure name or parameters ordering.*



​That's where the difference is. I think the client has to know the
procedure name and parameters. Otherwise​ we have a translation factory
pattern, that converts one naming convention to another. And you won't be
able to call any service API if there is no code in the middleware to
translate it to the service API procedure name and parameters. To avoid
this - we can use a transparent proxy model - direct mapping of a client
call to service API naming, which can be done if the client invokes the
methods with the names in the service API, so that the middleware will just
pass parameters, and will not translate. Instead of:


updating user data:

<client: POST /user/ > => <middleware: convert to
/keystone/update/ > => <keystone: update>

we may use:

<client: POST /keystone/{ver:=x.0}/{method:=update} > =>
<middleware: just forward to clients[ver].getattr("method")(**kwargs) >
=> <keystone: update>


​The idea here is that if we have keystone 4.0 client, ​we will have to
just add it to the clients [] list and nothing more is required at the
middleware level. Just create the frontend code to use the new Keystone 4.0
methods. Otherwise we will have to add all new/different signatures of 4.0
against 2.0/3.0 in the middleware in order to use Keystone 4.0.

There is also a great example of using a pluggable/new feature in Horizon.
Do you remember the volume types support patch? The patch was pending in
Gerrit for few months - first waiting the cinder support for volume types
to go upstream, then waiting few more weeks for review. I am not sure, but
as far as I remember, the Horizon patch even missed a release milestone and
was introduced in the next release.

If we have a transparent middleware - this will be no more an issue. As
long as someone has written the frontend modules(which should be easy to
add and customize), and they install the required version of the service
API - they will not need updated Horizon to start using the feature. Maybe
I am not the right person to give examples here, but how many of you had
some kind of Horizon customization being locally merged/patched in your
local distros/setups, until the patch is being pushed upstream?

I will say it again. Nova, Keystone, Cinder, Glance etc. already have
stable public APIs. Why do we want to add the translation middleware and to
introduce another level of REST API? This layer will often hide new
features, added to the service APIs and will delay their appearance in
Horizon. That's simply not needed. I believe it is possible to just wrap
the authentication in the middleware REST, but not to translate anything as
RPC methods/parameters.


​And one more example:

​@rest_utils.ajax()
def put(self, request, id):
"""Update a single project.

The POST data should be an application/json object containing the
parameters to update: "name" (string), "description" (string),
"domain_id" (string) and "enabled" (boolean, defaults to true).
Additional, undefined parameters may also be provided, but you'll
have
to look deep into keystone to figure out what they might be.

This method returns HTTP 204 (no content) on success.
"""
project = api.keystone.tenant_get(request, id)
kwargs = _tenant_kwargs_from_DATA(request.DATA, enabled=None)
api.keystone.tenant_update(request, project, **kwargs)


​Do we really need the lines:​

project = api.keystone.tenant_get(request, id)
kwargs = _tenant_kwargs_from_DATA(request.DATA, enabled=None)

​
? ​Since we update the project on the client, it is obvious that we already
fetched the project data. So we can simply send:


POST /keystone/3.0/tenant_update

Content-Type: application/json

{"id": cached.id, "domain_id": cached.domain_id, "name": "new name",
"description": "new description", "enabled": cached.enabled}

Fewer requests, faster application.




On Wed, Dec 10, 2014 at 8:39 PM, Thai Q Tran <***@us.ibm.com> wrote:

>
> ​​
> I think we're arguing for the same thing, but maybe slightly different
> approach. I think we can both agree that a middle-layer is required,
> whether we intend to use it as a proxy or REST endpoints. Regardless of the
> approach, the client needs to relay what API it wants to invoke, and you
> can do that either via RPC or REST. I personally prefer the REST approach
> because it shields the client. Client just needs to know which URL to hit
> in order to invoke a certain API, and does not need to know the procedure
> name or parameters ordering. Having said all of that, I do believe we
> should keep it as thin as possible. I do like the idea of having separate
> classes for different API versions. What we have today is a thin REST layer
> that acts like a proxy. You hit a certain URL, and the middle layer
> forwards the API invokation. The only exception to this rule is support for
> batch deletions.
>
> -----Tihomir Trifonov <***@gmail.com> wrote: -----
> To: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-***@lists.openstack.org>
> From: Tihomir Trifonov <***@gmail.com>
> Date: 12/10/2014 03:04AM
>
> Subject: Re: [openstack-dev] [horizon] REST and Django
>
> Richard, thanks for the reply,
>
>
> I agree that the given example is not a real REST. But we already have the
> REST API - that's Keystone, Nova, Cinder, Glance, Neutron etc, APIs. So
> what we plan to do here? To add a new REST layer to communicate with other
> REST API? Do we really need Frontend-REST-REST architecture ? My opinion is
> that we don't need another REST layer as we currently are trying to go away
> from the Django layer, which is the same - another processing layer.
> Although we call it REST proxy or whatever - it doesn't need to be a real
> REST, but just an aggregation proxy that combines and forwards some
> requests with adding minimal processing overhead. What makes sense for me
> is to keep the authentication in this layer as it is now - push a cookie to
> the frontend, but the REST layer will extract the auth tokens from the
> session storage and prepare the auth context for the REST API request to OS
> services. This way we will not expose the tokens to the JS frontend, and
> will have strict control over the authentication. The frontend will just
> send data requests, they will be wrapped with auth context and forwarded.
>
> Regarding the existing issues with versions in the API - for me the
> existing approach is wrong. All these fixes were made as workarounds. What
> should have been done is to create abstractions for each version and to use
> a separate class for each version. This was partially done for the
> keystoneclient in api/keystone.py, but not for the forms/views, where we
> still have if-else for versions. What I suggest here is to have
> different(concrete) views/forms for each version, and to use them according
> the context. If the Keystone backend is v2.0 - then in the Frontend use
> keystone2() object, otherwise use keystone3() object. This of course needs
> some more coding, but is much cleaner in terms of customization and
> testing. For me the current hacks with 'if keystone.version == 3.0' are
> wrong at many levels. And this can be solved now. *The problem till now
> was that we had one frontend that had to be backed by different versions of
> backend components*. *Now we can have different frontends that map to
> specific backend*. That's how I understand the power of Angular with it's
> views and directives. That's where I see the real benefit of using
> full-featured frontend. Also imagine how easy will be then to deprecate a
> component version, compared to what we need to do now for the same.
>
> Otherwise we just rewrite the current Django middleware with another
> DjangoRest middleware and don't change anything, we don't fix the problems.
> We just move them to another place.
>
> I still think that in Paris we talked about a new generation of the
> Dashboard, a different approach on building the frontend for OpenStack.
> What I've heard there from users/operators of Horizon was that it was
> extremely hard to add customizations and new features to the Dashboard, as
> all these needed to go through upstream changes and to wait until next
> release cycle to get them. Do we still want to address these concerns and
> how? Please, correct me if I got things wrong.
>
>
> On Wed, Dec 10, 2014 at 11:56 AM, Richard Jones <***@gmail.com>
> wrote:
>
>> Sorry I didn't respond to this earlier today, I had intended to.
>>
>> What you're describing isn't REST, and the principles of REST are what
>> have been guiding the design of the new API so far. I see a lot of value in
>> using REST approaches, mostly around clarity of the interface.
>>
>> While the idea of a very thin proxy seemed like a great idea at one
>> point, my conversations at the summit convinced me that there was value in
>> both using the client interfaces present in the openstack_dashboard/api
>> code base (since they abstract away many issues in the apis including
>> across versions) and also value in us being able to clean up (for example,
>> using "project_id" rather than "project" in the user API we've already
>> implemented) and extend those interfaces (to allow batched operations).
>>
>> We want to be careful about what we expose in Horizon to the JS clients
>> through this API. That necessitates some amount of code in Horizon. About
>> half of the current API for keysone represents that control (the other half
>> is docstrings :)
>>
>>
>> Richard
>>
>>
>> On Tue Dec 09 2014 at 9:37:47 PM Tihomir Trifonov <***@gmail.com>
>> wrote:
>>
>>> Sorry for the late reply, just few thoughts on the matter.
>>>
>>> IMO the REST middleware should be as thin as possible. And I mean thin
>>> in terms of processing - it should not do pre/post processing of the
>>> requests, but just unpack/pack. So here is an example:
>>>
>>> instead of making AJAX calls that contain instructions:
>>>
>>> ​​
>>>> POST --json --data {"action": "delete", "data": [ {"name":
>>>> "item1"}, {"name": "item2"}, {"name": "item3" ]}
>>>
>>>
>>> I think a better approach is just to pack/unpack batch commands, and
>>> leave execution to the frontend/backend and not middleware:
>>>
>>>
>>> ​​
>>>> POST --json --data {"
>>>> ​batch
>>>> ":
>>>> ​[
>>>> {​
>>>> "
>>>> ​
>>>> action"
>>>> ​ : "delete"​
>>>> ,
>>>> ​"payload": ​
>>>> {"name": "item1"}
>>>> ​,
>>>> {​
>>>> "
>>>> ​
>>>> action"
>>>> ​ : "delete"​
>>>> ,
>>>> ​
>>>> "payload":
>>>> ​
>>>> {"name": "item
>>>> ​2
>>>> "}
>>>> ​,
>>>> {​
>>>> "
>>>> ​
>>>> action"
>>>> ​ : "delete"​
>>>> ,
>>>> ​
>>>> "payload":
>>>> ​
>>>> {"name": "item
>>>> ​3
>>>> "}
>>>> ​ ] ​
>>>> ​
>>>> ​
>>>> }
>>>
>>>
>>> ​The idea is that the middleware should not know the actual data. It
>>> should ideally just unpack the data:
>>>
>>> ​​responses = []
>>>>
>>>
>>> for cmd in
>>>> ​ ​
>>>> ​
>>>> ​
>>>> request.POST['batch']:​
>>>
>>>
>>>> ​
>>>> ​​responses
>>>> ​.append(​
>>>> ​
>>>> getattr(controller, cmd['action']
>>>> ​)(**
>>>> cmd['​payload']
>>>> ​)​)
>>>>
>>>
>>>> ​return responses​
>>>>
>>>
>>>
>>> ​and the frontend(JS) will just send batches of simple commands, and
>>> will receive a list of responses for each command in the batch. The error
>>> handling will be done in the frontend​(JS) as well.
>>>
>>> ​
>>>
>>> For the more complex example of 'put()' where we have dependent objects:
>>>
>>> project = api.keystone.tenant_get(request, id)
>>>> kwargs = self._tenant_kwargs_from_DATA(request.DATA, enabled=None)
>>>> api.keystone.tenant_update(request, project, **kwargs)
>>>
>>>
>>>
>>> In practice the project data should be already present in the
>>> frontend(assuming that we already loaded it to render the project
>>> form/view), so
>>>
>>> ​
>>> ​
>>> POST --json --data {"
>>> ​batch
>>> ":
>>> ​[
>>> {​
>>> "
>>> ​
>>> action"
>>> ​ : "tenant_update"​
>>> ,
>>> ​"payload": ​
>>> {"project": js_project_object.id, "name": "some name", "prop1": "some
>>> prop", "prop2": "other prop, etc."}
>>> ​
>>> ​ ] ​
>>> ​
>>> ​
>>> }​
>>>
>>> So in general we don't need to recreate the full state on each REST
>>> call, if we make the Frontent full-featured application. This way - the
>>> frontend will construct the object, will hold the cached value, and will
>>> just send the needed requests as single ones or in batches, will receive
>>> the response from the API backend, and will render the results. The whole
>>> processing logic will be held in the Frontend(JS), while the middleware
>>> will just act as proxy(un/packer). This way we will maintain just the logic
>>> in the frontend, and will not need to duplicate some logic in the
>>> middleware.
>>>
>>>
>>>
>>>
>>> On Tue, Dec 2, 2014 at 4:45 PM, Adam Young <***@redhat.com> wrote:
>>>
>>>> On 12/02/2014 12:39 AM, Richard Jones wrote:
>>>>
>>>> On Mon Dec 01 2014 at 4:18:42 PM Thai Q Tran <***@us.ibm.com> wrote:
>>>>
>>>>> I agree that keeping the API layer thin would be ideal. I should add
>>>>> that having discrete API calls would allow dynamic population of table.
>>>>> However, I will make a case where it *might* be necessary to add
>>>>> additional APIs. Consider that you want to delete 3 items in a given table.
>>>>>
>>>>> If you do this on the client side, you would need to perform: n * (1
>>>>> API request + 1 AJAX request)
>>>>> If you have some logic on the server side that batch delete actions: n
>>>>> * (1 API request) + 1 AJAX request
>>>>>
>>>>> Consider the following:
>>>>> n = 1, client = 2 trips, server = 2 trips
>>>>> n = 3, client = 6 trips, server = 4 trips
>>>>> n = 10, client = 20 trips, server = 11 trips
>>>>> n = 100, client = 200 trips, server 101 trips
>>>>>
>>>>> As you can see, this does not scale very well.... something to
>>>>> consider...
>>>>>
>>>> This is not something Horizon can fix. Horizon can make matters
>>>> worse, but cannot make things better.
>>>>
>>>> If you want to delete 3 users, Horizon still needs to make 3 distinct
>>>> calls to Keystone.
>>>>
>>>> To fix this, we need either batch calls or a standard way to do
>>>> multiples of the same operation.
>>>>
>>>> The unified API effort it the right place to drive this.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Yep, though in the above cases the client is still going to be
>>>> hanging, waiting for those server-backend calls, with no feedback until
>>>> it's all done. I would hope that the client-server call overhead is
>>>> minimal, but I guess that's probably wishful thinking when in the land of
>>>> random Internet users hitting some provider's Horizon :)
>>>>
>>>> So yeah, having mulled it over myself I agree that it's useful to
>>>> have batch operations implemented in the POST handler, the most common
>>>> operation being DELETE.
>>>>
>>>> Maybe one day we could transition to a batch call with user feedback
>>>> using a websocket connection.
>>>>
>>>>
>>>> Richard
>>>>
>>>>> Richard Jones ---11/27/2014 05:38:53 PM---On Fri Nov 28 2014 at
>>>>> 5:58:00 AM Tripp, Travis S <***@hp.com> wrote:
>>>>>
>>>>> From: Richard Jones <***@gmail.com>
>>>>> To: "Tripp, Travis S" <***@hp.com>, OpenStack List <
>>>>> openstack-***@lists.openstack.org>
>>>>> Date: 11/27/2014 05:38 PM
>>>>> Subject: Re: [openstack-dev] [horizon] REST and Django
>>>>>
>>>>> ------------------------------
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Fri Nov 28 2014 at 5:58:00 AM Tripp, Travis S <
>>>>> ****@hp.com* <***@hp.com>> wrote:
>>>>>
>>>>> Hi Richard,
>>>>>
>>>>> You are right, we should put this out on the main ML, so copying
>>>>> thread out to there. ML: FYI that this started after some impromptu IRC
>>>>> discussions about a specific patch led into an impromptu google hangout
>>>>> discussion with all the people on the thread below.
>>>>>
>>>>>
>>>>> Thanks Travis!
>>>>>
>>>>>
>>>>>
>>>>> As I mentioned in the review[1], Thai and I were mainly discussing
>>>>> the possible performance implications of network hops from client to
>>>>> horizon server and whether or not any aggregation should occur server side.
>>>>> In other words, some views require several APIs to be queried before any
>>>>> data can displayed and it would eliminate some extra network requests from
>>>>> client to server if some of the data was first collected on the server side
>>>>> across service APIs. For example, the launch instance wizard will need to
>>>>> collect data from quite a few APIs before even the first step is displayed
>>>>> (I’ve listed those out in the blueprint [2]).
>>>>>
>>>>> The flip side to that (as you also pointed out) is that if we keep
>>>>> the API’s fine grained then the wizard will be able to optimize in one
>>>>> place the calls for data as it is needed. For example, the first step may
>>>>> only need half of the API calls. It also could lead to perceived
>>>>> performance increases just due to the wizard making a call for different
>>>>> data independently and displaying it as soon as it can.
>>>>>
>>>>>
>>>>> Indeed, looking at the current launch wizard code it seems like you
>>>>> wouldn't need to load all that data for the wizard to be displayed, since
>>>>> only some subset of it would be necessary to display any given panel of the
>>>>> wizard.
>>>>>
>>>>>
>>>>>
>>>>> I tend to lean towards your POV and starting with discrete API
>>>>> calls and letting the client optimize calls. If there are performance
>>>>> problems or other reasons then doing data aggregation on the server side
>>>>> could be considered at that point.
>>>>>
>>>>>
>>>>> I'm glad to hear it. I'm a fan of optimising when necessary, and not
>>>>> beforehand :)
>>>>>
>>>>>
>>>>>
>>>>> Of course if anybody is able to do some performance testing
>>>>> between the two approaches then that could affect the direction taken.
>>>>>
>>>>>
>>>>> I would certainly like to see us take some measurements when
>>>>> performance issues pop up. Optimising without solid metrics is bad idea :)
>>>>>
>>>>>
>>>>> Richard
>>>>>
>>>>>
>>>>>
>>>>> [1]
>>>>> *https://review.openstack.org/#/c/136676/8/openstack_dashboard/api/rest/urls.py*
>>>>> <https://review.openstack.org/#/c/136676/8/openstack_dashboard/api/rest/urls.py>
>>>>> [2]
>>>>> *https://blueprints.launchpad.net/horizon/+spec/launch-instance-redesign*
>>>>> <https://blueprints.launchpad.net/horizon/+spec/launch-instance-redesign>
>>>>>
>>>>> -Travis
>>>>>
>>>>> *From: *Richard Jones <****@gmail.com*
>>>>> <***@gmail.com>>
>>>>> * Date: *Wednesday, November 26, 2014 at 11:55 PM
>>>>> * To: *Travis Tripp <****@hp.com* <***@hp.com>>,
>>>>> Thai Q Tran/Silicon Valley/IBM <****@us.ibm.com*
>>>>> <***@us.ibm.com>>, David Lyle <****@gmail.com*
>>>>> <***@gmail.com>>, Maxime Vidori <****@enovance.com*
>>>>> <***@enovance.com>>, "Wroblewski, Szymon" <
>>>>> ****@intel.com* <***@intel.com>>,
>>>>> "Wood, Matthew David (HP Cloud - Horizon)" <****@hp.com*
>>>>> <***@hp.com>>, "Chen, Shaoquan" <****@hp.com*
>>>>> <***@hp.com>>, "Farina, Matt (HP Cloud)" <
>>>>> ****@hp.com* <***@hp.com>>, Cindy Lu/Silicon
>>>>> Valley/IBM <****@us.ibm.com* <***@us.ibm.com>>, Justin
>>>>> Pomeroy/Rochester/IBM <****@us.ibm.com* <***@us.ibm.com>>,
>>>>> Neill Cox <****@ingenious.com.au*
>>>>> <***@ingenious.com.au>>
>>>>> * Subject: *Re: REST and Django
>>>>>
>>>>> I'm not sure whether this is the appropriate place to discuss
>>>>> this, or whether I should be posting to the list under [Horizon] but I
>>>>> think we need to have a clear idea of what goes in the REST API and what
>>>>> goes in the client (angular) code.
>>>>>
>>>>> In my mind, the thinner the REST API the better. Indeed if we can
>>>>> get away with proxying requests through without touching any *client code,
>>>>> that would be great.
>>>>>
>>>>> Coding additional logic into the REST API means that a developer
>>>>> would need to look in two places, instead of one, to determine what was
>>>>> happening for a particular call. If we keep it thin then the API presented
>>>>> to the client developer is very, very similar to the API presented by the
>>>>> services. Minimum surprise.
>>>>>
>>>>> Your thoughts?
>>>>>
>>>>>
>>>>> Richard
>>>>>
>>>>>
>>>>> On Wed Nov 26 2014 at 2:40:52 PM Richard Jones <
>>>>> ****@gmail.com* <***@gmail.com>> wrote:
>>>>>
>>>>>
>>>>> Thanks for the great summary, Travis.
>>>>>
>>>>> I've completed the work I pledged this morning, so now the REST
>>>>> API change set has:
>>>>>
>>>>> - no rest framework dependency
>>>>> - AJAX scaffolding in openstack_dashboard.api.rest.utils
>>>>> - code in openstack_dashboard/api/rest/
>>>>> - renamed the API from "identity" to "keystone" to be consistent
>>>>> - added a sample of testing, mostly for my own sanity to check
>>>>> things were working
>>>>>
>>>>> *https://review.openstack.org/#/c/136676*
>>>>> <https://review.openstack.org/#/c/136676>
>>>>>
>>>>>
>>>>> Richard
>>>>>
>>>>> On Wed Nov 26 2014 at 12:18:25 PM Tripp, Travis S <
>>>>> ****@hp.com* <***@hp.com>> wrote:
>>>>>
>>>>>
>>>>> Hello all,
>>>>>
>>>>> Great discussion on the REST urls today! I think that we are
>>>>> on track to come to a common REST API usage pattern. To provide quick
>>>>> summary:
>>>>>
>>>>> We all agreed that going to a straight REST pattern rather
>>>>> than through tables was a good idea. We discussed using direct get / post
>>>>> in Django views like what Max originally used[1][2] and Thai also
>>>>> started[3] with the identity table rework or to go with djangorestframework
>>>>> [5] like what Richard was prototyping with[4].
>>>>>
>>>>> The main things we would use from Django Rest Framework were
>>>>> built in JSON serialization (avoid boilerplate), better exception handling,
>>>>> and some request wrapping. However, we all weren’t sure about the need for
>>>>> a full new framework just for that. At the end of the conversation, we
>>>>> decided that it was a cleaner approach, but Richard would see if he could
>>>>> provide some utility code to do that much for us without requiring the full
>>>>> framework. David voiced that he doesn’t want us building out a whole
>>>>> framework on our own either.
>>>>>
>>>>> So, Richard will do some investigation during his day today
>>>>> and get back to us. Whatever the case, we’ll get a patch in horizon for
>>>>> the base dependency (framework or Richard’s utilities) that both Thai’s
>>>>> work and the launch instance work is dependent upon. We’ll build REST
>>>>> style API’s using the same pattern. We will likely put the rest api’s in
>>>>> horizon/openstack_dashboard/api/rest/.
>>>>>
>>>>> [1]
>>>>> *https://review.openstack.org/#/c/133178/1/openstack_dashboard/workflow/keypair.py*
>>>>> <https://review.openstack.org/#/c/133178/1/openstack_dashboard/workflow/keypair.py>
>>>>> [2]
>>>>> *https://review.openstack.org/#/c/133178/1/openstack_dashboard/workflow/launch.py*
>>>>> <https://review.openstack.org/#/c/133178/1/openstack_dashboard/workflow/launch.py>
>>>>> [3]
>>>>> *https://review.openstack.org/#/c/133767/8/openstack_dashboard/dashboards/identity/users/views.py*
>>>>> <https://review.openstack.org/#/c/133767/8/openstack_dashboard/dashboards/identity/users/views.py>
>>>>> [4]
>>>>> *https://review.openstack.org/#/c/136676/4/openstack_dashboard/rest_api/identity.py*
>>>>> <https://review.openstack.org/#/c/136676/4/openstack_dashboard/rest_api/identity.py>
>>>>> [5] *http://www.django-rest-framework.org/*
>>>>> <http://www.django-rest-framework.org/>
>>>>>
>>>>> Thanks,
>>>>>
>>>>>
>>>>> Travis_______________________________________________
>>>>> OpenStack-dev mailing list
>>>>> OpenStack-***@lists.openstack.org
>>>>>
>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> OpenStack-dev mailing list
>>>>> OpenStack-***@lists.openstack.org
>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> OpenStack-dev mailing list
>>>> OpenStack-***@lists.openstack.org
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> OpenStack-dev mailing list
>>>> OpenStack-***@lists.openstack.org
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>>
>>>
>>>
>>> --
>>> Regards,
>>> Tihomir Trifonov
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-***@lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-***@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
>
> --
> Regards,
> Tihomir Trifonov
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-***@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-***@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


--
Regards,
Tihomir Trifonov
Tripp, Travis S
2014-12-12 02:02:30 UTC
Permalink
Tihomir,

Your comments in the patch were the actually the clearest to me about ease of customizing without requiring upstream changes and really made me think more about your points.

Here are a couple of bullet points for consideration.


* Will we take on auto-discovery of API extensions in two spots (python for legacy and JS for new)?
* As teams move towards deprecating / modifying APIs who will be responsible for ensuring the JS libraries stay up to date and keep tabs on every single project? Right now in Glance, for example, they are working on some fixes to v2 API (soon to become v2.3) that will allow them to deprecate v1 so that Nova can migrate from v1. Part of this includes making simultaneous improvements to the client library so that the switch can happen more transparently to client users. This testing and maintenance of the service team already takes on.
* The service API documentation almost always lags (helped by specs now) and the service team takes on the burden of exposing a programmatic way to access which is tested and easily consumable via the python clients which removes some guesswork from using the service.
* This is going to be an incremental approach with legacy support requirements anyway, I think. So, incorporating python side changes won’t just go away.
* A tangent that needs to be considered IMO since I’m working on some elastic search things right now. Which of these would be better if we introduce a server side caching mechanism or a new source of data such as elastic search to improve performance?
* Would the client just be able to handle changing whether or not it used cache with a header and in either case the server side appropriately uses the cache? (e.g. Cache-Control: no-cache)

I’m not sure I fully understood your example about Cinder. Was it the cinderclient that held up delivery of that horizon support or there cinder API or both? If the API isn’t in, then it would hold up delivery of the feature in any case. If it is just about delivering new functionality, all that would be required in Richard’s approach is to drop in a new file of decorated classes / functions from his utility with the API’s you want? None of the API calls have anything to do with how your view actually replaces the upstream view. These are all just about accessing the data.

Finally, I mentioned the following in the patch related to your example below about the client making two calls to do an update, but wanted to mention here to see if it is an approach that was purposeful (I don’t know the history):

​>>Do we really need the lines:​

>> project = api.keystone.tenant_get(request, id)
>> kwargs = _tenant_kwargs_from_DATA(request.DATA, enabled=None)
​
I agree that if you already have all the data it is really bad to have to do another call. I do think there is room for discussing the reasoning, though.
As far as I can tell, they do this so that if you are updating an entity, you have to be very specific about the fields you are changing. I actually see this as potentially a protectionary measure against data loss and a sometimes very nice to have feature. It perhaps was intended to *help* guard against race conditions *sometimes*.

Here's an example: Admin user Joe has an Domain open and stares at it for 15 minutes while he updates the description. Admin user Bob is asked to go ahead and enable it. He opens the record, edits it, and then saves it. Joe finished perfecting the description and saves it. Doing this action would mean that the Domain is enabled and the description gets updated. Last man in still wins if he updates the same fields, but if they update different fields then both of their changes will take affect without them stomping on each other. Whether that is good or bad may depend on the situation


From: Tihomir Trifonov <***@gmail.com<mailto:***@gmail.com>>
Reply-To: OpenStack List <openstack-***@lists.openstack.org<mailto:openstack-***@lists.openstack.org>>
Date: Thursday, December 11, 2014 at 7:53 AM
To: OpenStack List <openstack-***@lists.openstack.org<mailto:openstack-***@lists.openstack.org>>
Subject: Re: [openstack-dev] [horizon] REST and Django

​​
Client just needs to know which URL to hit in order to invoke a certain API, and does not need to know the procedure name or parameters ordering.


​That's where the difference is. I think the client has to know the procedure name and parameters. Otherwise​ we have a translation factory pattern, that converts one naming convention to another. And you won't be able to call any service API if there is no code in the middleware to translate it to the service API procedure name and parameters. To avoid this - we can use a transparent proxy model - direct mapping of a client call to service API naming, which can be done if the client invokes the methods with the names in the service API, so that the middleware will just pass parameters, and will not translate. Instead of:


updating user data:

<client: POST /user/ > => <middleware: convert to /keystone/update/ > => <keystone: update>

we may use:

<client: POST /keystone/{ver:=x.0}/{method:=update} > => <middleware: just forward to clients[ver].getattr("method")(**kwargs) > => <keystone: update>


​The idea here is that if we have keystone 4.0 client, ​we will have to just add it to the clients [] list and nothing more is required at the middleware level. Just create the frontend code to use the new Keystone 4.0 methods. Otherwise we will have to add all new/different signatures of 4.0 against 2.0/3.0 in the middleware in order to use Keystone 4.0.

There is also a great example of using a pluggable/new feature in Horizon. Do you remember the volume types support patch? The patch was pending in Gerrit for few months - first waiting the cinder support for volume types to go upstream, then waiting few more weeks for review. I am not sure, but as far as I remember, the Horizon patch even missed a release milestone and was introduced in the next release.

If we have a transparent middleware - this will be no more an issue. As long as someone has written the frontend modules(which should be easy to add and customize), and they install the required version of the service API - they will not need updated Horizon to start using the feature. Maybe I am not the right person to give examples here, but how many of you had some kind of Horizon customization being locally merged/patched in your local distros/setups, until the patch is being pushed upstream?

I will say it again. Nova, Keystone, Cinder, Glance etc. already have stable public APIs. Why do we want to add the translation middleware and to introduce another level of REST API? This layer will often hide new features, added to the service APIs and will delay their appearance in Horizon. That's simply not needed. I believe it is possible to just wrap the authentication in the middleware REST, but not to translate anything as RPC methods/parameters.


​And one more example:

​@rest_utils.ajax()
def put(self, request, id):
"""Update a single project.

The POST data should be an application/json object containing the
parameters to update: "name" (string), "description" (string),
"domain_id" (string) and "enabled" (boolean, defaults to true).
Additional, undefined parameters may also be provided, but you'll have
to look deep into keystone to figure out what they might be.

This method returns HTTP 204 (no content) on success.
"""
project = api.keystone.tenant_get(request, id)
kwargs = _tenant_kwargs_from_DATA(request.DATA, enabled=None)
api.keystone.tenant_update(request, project, **kwargs)

​Do we really need the lines:​

project = api.keystone.tenant_get(request, id)
kwargs = _tenant_kwargs_from_DATA(request.DATA, enabled=None)
​
? ​Since we update the project on the client, it is obvious that we already fetched the project data. So we can simply send:


POST /keystone/3.0/tenant_update

Content-Type: application/json

{"id": cached.id<http://cached.id>, "domain_id": cached.domain_id, "name": "new name", "description": "new description", "enabled": cached.enabled}

Fewer requests, faster application.




On Wed, Dec 10, 2014 at 8:39 PM, Thai Q Tran <***@us.ibm.com<mailto:***@us.ibm.com>> wrote:

​​
I think we're arguing for the same thing, but maybe slightly different approach. I think we can both agree that a middle-layer is required, whether we intend to use it as a proxy or REST endpoints. Regardless of the approach, the client needs to relay what API it wants to invoke, and you can do that either via RPC or REST. I personally prefer the REST approach because it shields the client. Client just needs to know which URL to hit in order to invoke a certain API, and does not need to know the procedure name or parameters ordering. Having said all of that, I do believe we should keep it as thin as possible. I do like the idea of having separate classes for different API versions. What we have today is a thin REST layer that acts like a proxy. You hit a certain URL, and the middle layer forwards the API invokation. The only exception to this rule is support for batch deletions.

-----Tihomir Trifonov <***@gmail.com<mailto:***@gmail.com>> wrote: -----
To: "OpenStack Development Mailing List (not for usage questions)" <openstack-***@lists.openstack.org<mailto:openstack-***@lists.openstack.org>>
From: Tihomir Trifonov <***@gmail.com<mailto:***@gmail.com>>
Date: 12/10/2014 03:04AM

Subject: Re: [openstack-dev] [horizon] REST and Django

Richard, thanks for the reply,


I agree that the given example is not a real REST. But we already have the REST API - that's Keystone, Nova, Cinder, Glance, Neutron etc, APIs. So what we plan to do here? To add a new REST layer to communicate with other REST API? Do we really need Frontend-REST-REST architecture ? My opinion is that we don't need another REST layer as we currently are trying to go away from the Django layer, which is the same - another processing layer. Although we call it REST proxy or whatever - it doesn't need to be a real REST, but just an aggregation proxy that combines and forwards some requests with adding minimal processing overhead. What makes sense for me is to keep the authentication in this layer as it is now - push a cookie to the frontend, but the REST layer will extract the auth tokens from the session storage and prepare the auth context for the REST API request to OS services. This way we will not expose the tokens to the JS frontend, and will have strict control over the authentication. The frontend will just send data requests, they will be wrapped with auth context and forwarded.

Regarding the existing issues with versions in the API - for me the existing approach is wrong. All these fixes were made as workarounds. What should have been done is to create abstractions for each version and to use a separate class for each version. This was partially done for the keystoneclient in api/keystone.py, but not for the forms/views, where we still have if-else for versions. What I suggest here is to have different(concrete) views/forms for each version, and to use them according the context. If the Keystone backend is v2.0 - then in the Frontend use keystone2() object, otherwise use keystone3() object. This of course needs some more coding, but is much cleaner in terms of customization and testing. For me the current hacks with 'if keystone.version == 3.0' are wrong at many levels. And this can be solved now. The problem till now was that we had one frontend that had to be backed by different versions of backend components. Now we can have different frontends that map to specific backend. That's how I understand the power of Angular with it's views and directives. That's where I see the real benefit of using full-featured frontend. Also imagine how easy will be then to deprecate a component version, compared to what we need to do now for the same.

Otherwise we just rewrite the current Django middleware with another DjangoRest middleware and don't change anything, we don't fix the problems. We just move them to another place.

I still think that in Paris we talked about a new generation of the Dashboard, a different approach on building the frontend for OpenStack. What I've heard there from users/operators of Horizon was that it was extremely hard to add customizations and new features to the Dashboard, as all these needed to go through upstream changes and to wait until next release cycle to get them. Do we still want to address these concerns and how? Please, correct me if I got things wrong.


On Wed, Dec 10, 2014 at 11:56 AM, Richard Jones <***@gmail.com<mailto:***@gmail.com>> wrote:
Sorry I didn't respond to this earlier today, I had intended to.

What you're describing isn't REST, and the principles of REST are what have been guiding the design of the new API so far. I see a lot of value in using REST approaches, mostly around clarity of the interface.

While the idea of a very thin proxy seemed like a great idea at one point, my conversations at the summit convinced me that there was value in both using the client interfaces present in the openstack_dashboard/api code base (since they abstract away many issues in the apis including across versions) and also value in us being able to clean up (for example, using "project_id" rather than "project" in the user API we've already implemented) and extend those interfaces (to allow batched operations).

We want to be careful about what we expose in Horizon to the JS clients through this API. That necessitates some amount of code in Horizon. About half of the current API for keysone represents that control (the other half is docstrings :)


Richard


On Tue Dec 09 2014 at 9:37:47 PM Tihomir Trifonov <***@gmail.com<mailto:***@gmail.com>> wrote:
Sorry for the late reply, just few thoughts on the matter.

IMO the REST middleware should be as thin as possible. And I mean thin in terms of processing - it should not do pre/post processing of the requests, but just unpack/pack. So here is an example:

instead of making AJAX calls that contain instructions:

​​
POST --json --data {"action": "delete", "data": [ {"name": "item1"}, {"name": "item2"}, {"name": "item3" ]}

I think a better approach is just to pack/unpack batch commands, and leave execution to the frontend/backend and not middleware:


​​
POST --json --data {"
​batch
":
​[
{​
"
​
action"
​ : "delete"​
,
​"payload": ​
{"name": "item1"}
​,
{​
"
​
action"
​ : "delete"​
,
​
"payload":
​
{"name": "item
​2
"}
​,
{​
"
​
action"
​ : "delete"​
,
​
"payload":
​
{"name": "item
​3
"}
​ ] ​
​
​
}

​The idea is that the middleware should not know the actual data. It should ideally just unpack the data:

​​responses = []

for cmd in
​ ​
​
​
request.POST['batch']:​

​
​​responses
​.append(​
​
getattr(controller, cmd['action']
​)(**
cmd['​payload']
​)​)

​return responses​


​and the frontend(JS) will just send batches of simple commands, and will receive a list of responses for each command in the batch. The error handling will be done in the frontend​(JS) as well.

​

For the more complex example of 'put()' where we have dependent objects:

project = api.keystone.tenant_get(request, id)
kwargs = self._tenant_kwargs_from_DATA(request.DATA, enabled=None)
api.keystone.tenant_update(request, project, **kwargs)


In practice the project data should be already present in the frontend(assuming that we already loaded it to render the project form/view), so

​
​
POST --json --data {"
​batch
":
​[
{​
"
​
action"
​ : "tenant_update"​
,
​"payload": ​
{"project": js_project_object.id<http://js_project_object.id>, "name": "some name", "prop1": "some prop", "prop2": "other prop, etc."}
​
​ ] ​
​
​
}​

So in general we don't need to recreate the full state on each REST call, if we make the Frontent full-featured application. This way - the frontend will construct the object, will hold the cached value, and will just send the needed requests as single ones or in batches, will receive the response from the API backend, and will render the results. The whole processing logic will be held in the Frontend(JS), while the middleware will just act as proxy(un/packer). This way we will maintain just the logic in the frontend, and will not need to duplicate some logic in the middleware.




On Tue, Dec 2, 2014 at 4:45 PM, Adam Young <***@redhat.com<mailto:***@redhat.com>> wrote:
On 12/02/2014 12:39 AM, Richard Jones wrote:
On Mon Dec 01 2014 at 4:18:42 PM Thai Q Tran <***@us.ibm.com<mailto:***@us.ibm.com>> wrote:

I agree that keeping the API layer thin would be ideal. I should add that having discrete API calls would allow dynamic population of table. However, I will make a case where it might be necessary to add additional APIs. Consider that you want to delete 3 items in a given table.

If you do this on the client side, you would need to perform: n * (1 API request + 1 AJAX request)
If you have some logic on the server side that batch delete actions: n * (1 API request) + 1 AJAX request

Consider the following:
n = 1, client = 2 trips, server = 2 trips
n = 3, client = 6 trips, server = 4 trips
n = 10, client = 20 trips, server = 11 trips
n = 100, client = 200 trips, server 101 trips

As you can see, this does not scale very well.... something to consider...

This is not something Horizon can fix. Horizon can make matters worse, but cannot make things better.

If you want to delete 3 users, Horizon still needs to make 3 distinct calls to Keystone.

To fix this, we need either batch calls or a standard way to do multiples of the same operation.

The unified API effort it the right place to drive this.







Yep, though in the above cases the client is still going to be hanging, waiting for those server-backend calls, with no feedback until it's all done. I would hope that the client-server call overhead is minimal, but I guess that's probably wishful thinking when in the land of random Internet users hitting some provider's Horizon :)

So yeah, having mulled it over myself I agree that it's useful to have batch operations implemented in the POST handler, the most common operation being DELETE.

Maybe one day we could transition to a batch call with user feedback using a websocket connection.


Richard

[cid:994faa67a8e28335_0.0.1.1]Richard Jones ---11/27/2014 05:38:53 PM---On Fri Nov 28 2014 at 5:58:00 AM Tripp, Travis S <***@hp.com<mailto:***@hp.com>> wrote:

From: Richard Jones <***@gmail.com<mailto:***@gmail.com>>
To: "Tripp, Travis S" <***@hp.com<mailto:***@hp.com>>, OpenStack List <openstack-***@lists.openstack.org<mailto:openstack-***@lists.openstack.org>>
Date: 11/27/2014 05:38 PM
Subject: Re: [openstack-dev] [horizon] REST and Django

________________________________



On Fri Nov 28 2014 at 5:58:00 AM Tripp, Travis S <***@hp.com<mailto:***@hp.com>> wrote:

Hi Richard,

You are right, we should put this out on the main ML, so copying thread out to there. ML: FYI that this started after some impromptu IRC discussions about a specific patch led into an impromptu google hangout discussion with all the people on the thread below.

Thanks Travis!



As I mentioned in the review[1], Thai and I were mainly discussing the possible performance implications of network hops from client to horizon server and whether or not any aggregation should occur server side. In other words, some views require several APIs to be queried before any data can displayed and it would eliminate some extra network requests from client to server if some of the data was first collected on the server side across service APIs. For example, the launch instance wizard will need to collect data from quite a few APIs before even the first step is displayed (I’ve listed those out in the blueprint [2]).

The flip side to that (as you also pointed out) is that if we keep the API’s fine grained then the wizard will be able to optimize in one place the calls for data as it is needed. For example, the first step may only need half of the API calls. It also could lead to perceived performance increases just due to the wizard making a call for different data independently and displaying it as soon as it can.

Indeed, looking at the current launch wizard code it seems like you wouldn't need to load all that data for the wizard to be displayed, since only some subset of it would be necessary to display any given panel of the wizard.



I tend to lean towards your POV and starting with discrete API calls and letting the client optimize calls. If there are performance problems or other reasons then doing data aggregation on the server side could be considered at that point.

I'm glad to hear it. I'm a fan of optimising when necessary, and not beforehand :)



Of course if anybody is able to do some performance testing between the two approaches then that could affect the direction taken.

I would certainly like to see us take some measurements when performance issues pop up. Optimising without solid metrics is bad idea :)


Richard


[1] https://review.openstack.org/#/c/136676/8/openstack_dashboard/api/rest/urls.py
[2] https://blueprints.launchpad.net/horizon/+spec/launch-instance-redesign

-Travis

From: Richard Jones <***@gmail.com<mailto:***@gmail.com>>
Date: Wednesday, November 26, 2014 at 11:55 PM
To: Travis Tripp <***@hp.com<mailto:***@hp.com>>, Thai Q Tran/Silicon Valley/IBM <***@us.ibm.com<mailto:***@us.ibm.com>>, David Lyle <***@gmail.com<mailto:***@gmail.com>>, Maxime Vidori <***@enovance.com<mailto:***@enovance.com>>, "Wroblewski, Szymon" <***@intel.com<mailto:***@intel.com>>, "Wood, Matthew David (HP Cloud - Horizon)" <***@hp.com<mailto:***@hp.com>>, "Chen, Shaoquan" <***@hp.com<mailto:***@hp.com>>, "Farina, Matt (HP Cloud)" <***@hp.com<mailto:***@hp.com>>, Cindy Lu/Silicon Valley/IBM <***@us.ibm.com<mailto:***@us.ibm.com>>, Justin Pomeroy/Rochester/IBM <***@us.ibm.com<mailto:***@us.ibm.com>>, Neill Cox <***@ingenious.com.au<mailto:***@ingenious.com.au>>
Subject: Re: REST and Django

I'm not sure whether this is the appropriate place to discuss this, or whether I should be posting to the list under [Horizon] but I think we need to have a clear idea of what goes in the REST API and what goes in the client (angular) code.

In my mind, the thinner the REST API the better. Indeed if we can get away with proxying requests through without touching any *client code, that would be great.

Coding additional logic into the REST API means that a developer would need to look in two places, instead of one, to determine what was happening for a particular call. If we keep it thin then the API presented to the client developer is very, very similar to the API presented by the services. Minimum surprise.

Your thoughts?


Richard


On Wed Nov 26 2014 at 2:40:52 PM Richard Jones <***@gmail.com<mailto:***@gmail.com>> wrote:

Thanks for the great summary, Travis.

I've completed the work I pledged this morning, so now the REST API change set has:

- no rest framework dependency
- AJAX scaffolding in openstack_dashboard.api.rest.utils
- code in openstack_dashboard/api/rest/
- renamed the API from "identity" to "keystone" to be consistent
- added a sample of testing, mostly for my own sanity to check things were working

https://review.openstack.org/#/c/136676


Richard

On Wed Nov 26 2014 at 12:18:25 PM Tripp, Travis S <***@hp.com<mailto:***@hp.com>> wrote:

Hello all,

Great discussion on the REST urls today! I think that we are on track to come to a common REST API usage pattern. To provide quick summary:

We all agreed that going to a straight REST pattern rather than through tables was a good idea. We discussed using direct get / post in Django views like what Max originally used[1][2] and Thai also started[3] with the identity table rework or to go with djangorestframework [5] like what Richard was prototyping with[4].

The main things we would use from Django Rest Framework were built in JSON serialization (avoid boilerplate), better exception handling, and some request wrapping. However, we all weren’t sure about the need for a full new framework just for that. At the end of the conversation, we decided that it was a cleaner approach, but Richard would see if he could provide some utility code to do that much for us without requiring the full framework. David voiced that he doesn’t want us building out a whole framework on our own either.

So, Richard will do some investigation during his day today and get back to us. Whatever the case, we’ll get a patch in horizon for the base dependency (framework or Richard’s utilities) that both Thai’s work and the launch instance work is dependent upon. We’ll build REST style API’s using the same pattern. We will likely put the rest api’s in horizon/openstack_dashboard/api/rest/.

[1] https://review.openstack.org/#/c/133178/1/openstack_dashboard/workflow/keypair.py
[2] https://review.openstack.org/#/c/133178/1/openstack_dashboard/workflow/launch.py
[3] https://review.openstack.org/#/c/133767/8/openstack_dashboard/dashboards/identity/users/views.py
[4] https://review.openstack.org/#/c/136676/4/openstack_dashboard/rest_api/identity.py
[5] http://www.django-rest-framework.org/

Thanks,

Travis_______________________________________________
OpenStack-dev mailing list
OpenStack-***@lists.openstack.org<mailto:OpenStack-***@lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


_______________________________________________
OpenStack-dev mailing list
OpenStack-***@lists.openstack.org<mailto:OpenStack-***@lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


_______________________________________________
OpenStack-dev mailing list
OpenStack-***@lists.openstack.org<mailto:OpenStack-***@lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


_______________________________________________
OpenStack-dev mailing list
OpenStack-***@lists.openstack.org<mailto:OpenStack-***@lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




--
Regards,
Tihomir Trifonov
_______________________________________________
OpenStack-dev mailing list
OpenStack-***@lists.openstack.org<mailto:OpenStack-***@lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

_______________________________________________
OpenStack-dev mailing list
OpenStack-***@lists.openstack.org<mailto:OpenStack-***@lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




--
Regards,
Tihomir Trifonov
_______________________________________________
OpenStack-dev mailing list
OpenStack-***@lists.openstack.org<mailto:OpenStack-***@lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

_______________________________________________
OpenStack-dev mailing list
OpenStack-***@lists.openstack.org<mailto:OpenStack-***@lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




--
Regards,
Tihomir Trifonov
Richard Jones
2014-12-12 02:30:03 UTC
Permalink
On Fri Dec 12 2014 at 1:06:08 PM Tripp, Travis S <***@hp.com>
wrote:

> ​>>Do we really need the lines:​
>
> >> project = api.keystone.tenant_get(request, id)
> >> kwargs = _tenant_kwargs_from_DATA(request.DATA, enabled=None)
> ​
> I agree that if you already have all the data it is really bad to have to
> do another call. I do think there is room for discussing the reasoning,
> though.
> As far as I can tell, they do this so that if you are updating an entity,
> you have to be very specific about the fields you are changing. I actually
> see this as potentially a protectionary measure against data loss and a
> sometimes very nice to have feature. It perhaps was intended to *help*
> guard against race conditions *sometimes*.
>

Yep, it looks like I broke this API by implementing it the way I did, and
I'll alter the API so that you pass both the "current" object (according to
the client) and the parameters to alter.

Thanks everyone for the great reviewing!


Richard
Tripp, Travis S
2014-12-12 06:08:13 UTC
Permalink
I just re-read and I apologize for the hastily written email I previously
sent. I’ll try to salvage it with a bit of a revision below (please ignore
the previous email).

On 12/11/14, 7:02 PM, "Tripp, Travis S" <***@hp.com> wrote
(REVISED):

>Tihomir,
>
>Your comments in the patch were very helpful for me to understand your
>concerns about the ease of customizing without requiring upstream
>changes. It also reminded me that I’ve also previously questioned the
>python middleman.
>
>However, here are a couple of bullet points for Devil’s Advocate
>consideration.
>
>
> * Will we take on auto-discovery of API extensions in two spots
>(python for legacy and JS for new)?
> * The Horizon team will have to keep an even closer eye on every
>single project and be ready to react if there are changes to the API that
>break things. Right now in Glance, for example, they are working on some
>fixes to the v2 API (soon to become v2.3) that will allow them to
>deprecate v1 somewhat transparently to users of the client library.
> * The service API documentation almost always lags (although, helped
>by specs now) and the service team takes on the burden of exposing a
>programmatic way to access the API. This is tested and easily consumable
>via the python clients, which removes some guesswork from using the
>service.
> * This is going to be an incremental approach with legacy support
>requirements anyway. So, incorporating python side changes won’t just go
>away.
> * Which approach would be better if we introduce a server side
>caching mechanism or a new source of data such as elastic search to
>improve performance? Would the client side code have to be changed
>dramatically to take advantage of those improvements or could it be done
>transparently on the server side if we own the exposed API?
>
>I’m not sure I fully understood your example about Cinder. Was it the
>cinder client that held up delivery of horizon support, the cinder API or
>both? If the API isn’t in, then it would hold up delivery of the feature
>in any case. There still would be timing pressures to react and build a
>new view that supports it. For customization, with Richard’s approach new
>views could be supported by just dropping in a new REST API decorated
>module with the APIs you want, including direct pass through support if
>desired to new APIs. Downstream customizations / Upstream changes to
>views seem a bit like a bit of a related, but different issue to me as
>long as their is an easy way to drop in new API support.
>
>Finally, regarding the client making two calls to do an update:
>
>​>>Do we really need the lines:​
>
>>> project = api.keystone.tenant_get(request, id)
>>> kwargs = _tenant_kwargs_from_DATA(request.DATA, enabled=None)
>​
>I agree that if you already have all the data it may be bad to have to do
>another call. I do think there is room for discussing the reasoning,
>though.
>As far as I can tell, they do this so that if you are updating an entity,
>you have to be very specific about the fields you are changing. I
>actually see this as potentially a protectionary measure against data
>loss and sometimes a very nice to have feature. It perhaps was intended
>to *help* guard against race conditions (no locking and no transactions
>with many users simultaneously accessing the data).
>
>Here's an example: Admin user Joe has a Domain open and stares at it for
>15 minutes while he updates just the description. Admin user Bob is asked
>to go ahead and enable it. He opens the record, edits it, and then saves
>it. Joe finished perfecting the description and saves it. They could in
>effect both edit the same domain independently. Last man in still wins if
>he updates the same fields, but if they update different fields then both
>of their changes will take affect without them stomping on each other. Or
>maybe it is intended to encourage client users to compare their current
>and previous to see if they should issue a warning if the data changed
>between getting and updating the data. Or maybe like you said, it is just
>overhead API calls.



>
>From: Tihomir Trifonov <***@gmail.com<mailto:***@gmail.com>>
>Reply-To: OpenStack List
><openstack-***@lists.openstack.org<mailto:openstack-***@lists.openstack.or
>g>>
>Date: Thursday, December 11, 2014 at 7:53 AM
>To: OpenStack List
><openstack-***@lists.openstack.org<mailto:openstack-***@lists.openstack.or
>g>>
>Subject: Re: [openstack-dev] [horizon] REST and Django
>
>​​
>Client just needs to know which URL to hit in order to invoke a certain
>API, and does not need to know the procedure name or parameters ordering.
>
>
>​That's where the difference is. I think the client has to know the
>procedure name and parameters. Otherwise​ we have a translation factory
>pattern, that converts one naming convention to another. And you won't be
>able to call any service API if there is no code in the middleware to
>translate it to the service API procedure name and parameters. To avoid
>this - we can use a transparent proxy model - direct mapping of a client
>call to service API naming, which can be done if the client invokes the
>methods with the names in the service API, so that the middleware will
>just pass parameters, and will not translate. Instead of:
>
>
>updating user data:
>
> <client: POST /user/ > => <middleware: convert to
>/keystone/update/ > => <keystone: update>
>
>we may use:
>
> <client: POST /keystone/{ver:=x.0}/{method:=update} > =>
><middleware: just forward to clients[ver].getattr("method")(**kwargs) >
>=> <keystone: update>
>
>
>​The idea here is that if we have keystone 4.0 client, ​we will have to
>just add it to the clients [] list and nothing more is required at the
>middleware level. Just create the frontend code to use the new Keystone
>4.0 methods. Otherwise we will have to add all new/different signatures
>of 4.0 against 2.0/3.0 in the middleware in order to use Keystone 4.0.
>
>There is also a great example of using a pluggable/new feature in
>Horizon. Do you remember the volume types support patch? The patch was
>pending in Gerrit for few months - first waiting the cinder support for
>volume types to go upstream, then waiting few more weeks for review. I am
>not sure, but as far as I remember, the Horizon patch even missed a
>release milestone and was introduced in the next release.
>
>If we have a transparent middleware - this will be no more an issue. As
>long as someone has written the frontend modules(which should be easy to
>add and customize), and they install the required version of the service
>API - they will not need updated Horizon to start using the feature.
>Maybe I am not the right person to give examples here, but how many of
>you had some kind of Horizon customization being locally merged/patched
>in your local distros/setups, until the patch is being pushed upstream?
>
>I will say it again. Nova, Keystone, Cinder, Glance etc. already have
>stable public APIs. Why do we want to add the translation middleware and
>to introduce another level of REST API? This layer will often hide new
>features, added to the service APIs and will delay their appearance in
>Horizon. That's simply not needed. I believe it is possible to just wrap
>the authentication in the middleware REST, but not to translate anything
>as RPC methods/parameters.
>
>
>​And one more example:
>
>​@rest_utils.ajax()
>def put(self, request, id):
> """Update a single project.
>
> The POST data should be an application/json object containing the
> parameters to update: "name" (string), "description" (string),
> "domain_id" (string) and "enabled" (boolean, defaults to true).
> Additional, undefined parameters may also be provided, but you'll
>have
> to look deep into keystone to figure out what they might be.
>
> This method returns HTTP 204 (no content) on success.
> """
> project = api.keystone.tenant_get(request, id)
> kwargs = _tenant_kwargs_from_DATA(request.DATA, enabled=None)
> api.keystone.tenant_update(request, project, **kwargs)
>
>​Do we really need the lines:​
>
>project = api.keystone.tenant_get(request, id)
>kwargs = _tenant_kwargs_from_DATA(request.DATA, enabled=None)
>​
>? ​Since we update the project on the client, it is obvious that we
>already fetched the project data. So we can simply send:
>
>
>POST /keystone/3.0/tenant_update
>
>Content-Type: application/json
>
>{"id": cached.id<http://cached.id>, "domain_id": cached.domain_id,
>"name": "new name", "description": "new description", "enabled":
>cached.enabled}
>
>Fewer requests, faster application.
>
>
Tihomir Trifonov
2014-12-12 12:51:35 UTC
Permalink
>
> Here's an example: Admin user Joe has an Domain open and stares at it for
> 15 minutes while he updates the description. Admin user Bob is asked to go
> ahead and enable it. He opens the record, edits it, and then saves it. Joe
> finished perfecting the description and saves it. Doing this action would
> mean that the Domain is enabled and the description gets updated. Last man
> in still wins if he updates the same fields, but if they update different
> fields then both of their changes will take affect without them stomping on
> each other. Whether that is good or bad may depend on the situation




That's a great example. I believe that all of the Openstack APIs support
PATCH updates of arbitrary fields. This way - the frontend(AngularJS) can
detect which fields are being modified, and to submit only these fields for
update. If we however use a form with POST, although we will load the
object before updating it, the middleware cannot find which fields are
actually modified, and will update them all, which is more likely what PUT
should do. Thus having full control in the frontend part, we can submit
only changed fields. If however a service API doesn't support PATCH, it is
actually a problem in the API and not in the client...



The service API documentation almost always lags (although, helped by specs
> now) and the service team takes on the burden of exposing a programmatic
> way to access the API. This is tested and easily consumable via the
> python clients, which removes some guesswork from using the service.


True. But what if the service team modifies a method signature from let's
say:

def add_something(self, request,
> ​ field1, field2):
>
>
to

def add_something(self, request,
> ​ field1, field2, field3):
>
>
and in the middleware we have the old signature:

​def add_something(self, request,
> ​ field1, field2):
>

we still need to modify the middleware to add the new field. If however the
middleware is transparent and just passes **kwargs, it will pass through
whatever the frontend sends. So we just need to update the frontend, which
can be done using custom views, and not necessary going through an upstream
change. My point is why do we need to hide some features of the backend
service API behind a "firewall" what the middleware in fact is?







On Fri, Dec 12, 2014 at 8:08 AM, Tripp, Travis S <***@hp.com>
wrote:
>
> I just re-read and I apologize for the hastily written email I previously
> sent. I’ll try to salvage it with a bit of a revision below (please ignore
> the previous email).
>
> On 12/11/14, 7:02 PM, "Tripp, Travis S" <***@hp.com> wrote
> (REVISED):
>
> >Tihomir,
> >
> >Your comments in the patch were very helpful for me to understand your
> >concerns about the ease of customizing without requiring upstream
> >changes. It also reminded me that I’ve also previously questioned the
> >python middleman.
> >
> >However, here are a couple of bullet points for Devil’s Advocate
> >consideration.
> >
> >
> > * Will we take on auto-discovery of API extensions in two spots
> >(python for legacy and JS for new)?
> > * The Horizon team will have to keep an even closer eye on every
> >single project and be ready to react if there are changes to the API that
> >break things. Right now in Glance, for example, they are working on some
> >fixes to the v2 API (soon to become v2.3) that will allow them to
> >deprecate v1 somewhat transparently to users of the client library.
> > * The service API documentation almost always lags (although, helped
> >by specs now) and the service team takes on the burden of exposing a
> >programmatic way to access the API. This is tested and easily consumable
> >via the python clients, which removes some guesswork from using the
> >service.
> > * This is going to be an incremental approach with legacy support
> >requirements anyway. So, incorporating python side changes won’t just go
> >away.
> > * Which approach would be better if we introduce a server side
> >caching mechanism or a new source of data such as elastic search to
> >improve performance? Would the client side code have to be changed
> >dramatically to take advantage of those improvements or could it be done
> >transparently on the server side if we own the exposed API?
> >
> >I’m not sure I fully understood your example about Cinder. Was it the
> >cinder client that held up delivery of horizon support, the cinder API or
> >both? If the API isn’t in, then it would hold up delivery of the feature
> >in any case. There still would be timing pressures to react and build a
> >new view that supports it. For customization, with Richard’s approach new
> >views could be supported by just dropping in a new REST API decorated
> >module with the APIs you want, including direct pass through support if
> >desired to new APIs. Downstream customizations / Upstream changes to
> >views seem a bit like a bit of a related, but different issue to me as
> >long as their is an easy way to drop in new API support.
> >
> >Finally, regarding the client making two calls to do an update:
> >
> >​>>Do we really need the lines:​
> >
> >>> project = api.keystone.tenant_get(request, id)
> >>> kwargs = _tenant_kwargs_from_DATA(request.DATA, enabled=None)
> >​
> >I agree that if you already have all the data it may be bad to have to do
> >another call. I do think there is room for discussing the reasoning,
> >though.
> >As far as I can tell, they do this so that if you are updating an entity,
> >you have to be very specific about the fields you are changing. I
> >actually see this as potentially a protectionary measure against data
> >loss and sometimes a very nice to have feature. It perhaps was intended
> >to *help* guard against race conditions (no locking and no transactions
> >with many users simultaneously accessing the data).
> >
> >Here's an example: Admin user Joe has a Domain open and stares at it for
> >15 minutes while he updates just the description. Admin user Bob is asked
> >to go ahead and enable it. He opens the record, edits it, and then saves
> >it. Joe finished perfecting the description and saves it. They could in
> >effect both edit the same domain independently. Last man in still wins if
> >he updates the same fields, but if they update different fields then both
> >of their changes will take affect without them stomping on each other. Or
> >maybe it is intended to encourage client users to compare their current
> >and previous to see if they should issue a warning if the data changed
> >between getting and updating the data. Or maybe like you said, it is just
> >overhead API calls.
>
>
>
> >
> >From: Tihomir Trifonov <***@gmail.com<mailto:***@gmail.com
> >>
> >Reply-To: OpenStack List
> ><openstack-***@lists.openstack.org<mailto:
> openstack-***@lists.openstack.or
> >g>>
> >Date: Thursday, December 11, 2014 at 7:53 AM
> >To: OpenStack List
> ><openstack-***@lists.openstack.org<mailto:
> openstack-***@lists.openstack.or
> >g>>
> >Subject: Re: [openstack-dev] [horizon] REST and Django
> >
> >​​
> >Client just needs to know which URL to hit in order to invoke a certain
> >API, and does not need to know the procedure name or parameters ordering.
> >
> >
> >​That's where the difference is. I think the client has to know the
> >procedure name and parameters. Otherwise​ we have a translation factory
> >pattern, that converts one naming convention to another. And you won't be
> >able to call any service API if there is no code in the middleware to
> >translate it to the service API procedure name and parameters. To avoid
> >this - we can use a transparent proxy model - direct mapping of a client
> >call to service API naming, which can be done if the client invokes the
> >methods with the names in the service API, so that the middleware will
> >just pass parameters, and will not translate. Instead of:
> >
> >
> >updating user data:
> >
> > <client: POST /user/ > => <middleware: convert to
> >/keystone/update/ > => <keystone: update>
> >
> >we may use:
> >
> > <client: POST /keystone/{ver:=x.0}/{method:=update} > =>
> ><middleware: just forward to clients[ver].getattr("method")(**kwargs) >
> >=> <keystone: update>
> >
> >
> >​The idea here is that if we have keystone 4.0 client, ​we will have to
> >just add it to the clients [] list and nothing more is required at the
> >middleware level. Just create the frontend code to use the new Keystone
> >4.0 methods. Otherwise we will have to add all new/different signatures
> >of 4.0 against 2.0/3.0 in the middleware in order to use Keystone 4.0.
> >
> >There is also a great example of using a pluggable/new feature in
> >Horizon. Do you remember the volume types support patch? The patch was
> >pending in Gerrit for few months - first waiting the cinder support for
> >volume types to go upstream, then waiting few more weeks for review. I am
> >not sure, but as far as I remember, the Horizon patch even missed a
> >release milestone and was introduced in the next release.
> >
> >If we have a transparent middleware - this will be no more an issue. As
> >long as someone has written the frontend modules(which should be easy to
> >add and customize), and they install the required version of the service
> >API - they will not need updated Horizon to start using the feature.
> >Maybe I am not the right person to give examples here, but how many of
> >you had some kind of Horizon customization being locally merged/patched
> >in your local distros/setups, until the patch is being pushed upstream?
> >
> >I will say it again. Nova, Keystone, Cinder, Glance etc. already have
> >stable public APIs. Why do we want to add the translation middleware and
> >to introduce another level of REST API? This layer will often hide new
> >features, added to the service APIs and will delay their appearance in
> >Horizon. That's simply not needed. I believe it is possible to just wrap
> >the authentication in the middleware REST, but not to translate anything
> >as RPC methods/parameters.
> >
> >
> >​And one more example:
> >
> >​@rest_utils.ajax()
> >def put(self, request, id):
> > """Update a single project.
> >
> > The POST data should be an application/json object containing the
> > parameters to update: "name" (string), "description" (string),
> > "domain_id" (string) and "enabled" (boolean, defaults to true).
> > Additional, undefined parameters may also be provided, but you'll
> >have
> > to look deep into keystone to figure out what they might be.
> >
> > This method returns HTTP 204 (no content) on success.
> > """
> > project = api.keystone.tenant_get(request, id)
> > kwargs = _tenant_kwargs_from_DATA(request.DATA, enabled=None)
> > api.keystone.tenant_update(request, project, **kwargs)
> >
> >​Do we really need the lines:​
> >
> >project = api.keystone.tenant_get(request, id)
> >kwargs = _tenant_kwargs_from_DATA(request.DATA, enabled=None)
> >​
> >? ​Since we update the project on the client, it is obvious that we
> >already fetched the project data. So we can simply send:
> >
> >
> >POST /keystone/3.0/tenant_update
> >
> >Content-Type: application/json
> >
> >{"id": cached.id<http://cached.id>, "domain_id": cached.domain_id,
> >"name": "new name", "description": "new description", "enabled":
> >cached.enabled}
> >
> >Fewer requests, faster application.
> >
> >
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-***@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>


--
Regards,
Tihomir Trifonov
Thai Q Tran
2014-12-12 18:05:00 UTC
Permalink
_______________________________________________
OpenStack-dev mailing list
OpenStack-***@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Tripp, Travis S
2014-12-12 23:09:36 UTC
Permalink
Tihomir,

Today I added one glance call based on Richard’s decorator pattern[1] and started to play with incorporating some of your ideas. Please note, I only had limited time today. That is passing the kwargs through to the glance client. This was an interesting first choice, because it immediately highlighted a concrete example of the horizon glance wrapper post-processing still being useful (rather than be a direct pass-through with no wrapper). See below. If you have some some concrete code examples of your ideas it would be helpful.

[1] https://review.openstack.org/#/c/141273/2/openstack_dashboard/api/rest/glance.py

With the patch, basically, you can call the following and all of the GET parameters get passed directly through to the horizon glance client and you get results back as expected.

http://localhost:8002/api/glance/images/?sort_dir=desc&sort_key=created_at&paginate=True&marker=bb2cfb1c-2234-4f54-aec5-b4916fe2d747

If you pass in an incorrect sort_key, the glance client returns the following error message which propagates back to the REST caller as an error with the message:

"sort_key must be one of the following: name, status, container_format, disk_format, size, id, created_at, updated_at."

This is done by passing **request.GET.dict() through.

Please note, that if you try this (with POSTMAN, for example), you need to set the header of X-Requested-With = XMLHttpRequest

So, what issues did it immediately call out with directly invoking the client?

The python-glanceclient internally handles pagination by returning a generator. Each iteration on the generator will handle making a request for the next page of data. If you were to just do something like return list(image_generator) to serialize it back out to the caller, it would actually end up making a call back to the server X times to fetch all pages before serializing back (thereby not really paginating). The horizon glance client wrapper today handles this by using islice intelligently along with honoring the API_RESULT_LIMIT setting in Horizon. So, this gives a direct example of where the wrapper does something that a direct passthrough to the client would not allow.

That said, I can see a few ways that we could use the same REST decorator code and provide direct access to the API. We’d simply provide a class where the url_regex maps to the desired path and gives direct passthrough. Maybe that kind of passthrough could always be provided for ease of customization / extensibility and additional methods with wrappers provided when necessary. I need to leave for today, so can’t actually try that out at the moment.

Thanks,
Travis

From: Thai Q Tran <***@us.ibm.com<mailto:***@us.ibm.com>>
Reply-To: OpenStack List <openstack-***@lists.openstack.org<mailto:openstack-***@lists.openstack.org>>
Date: Friday, December 12, 2014 at 11:05 AM
To: OpenStack List <openstack-***@lists.openstack.org<mailto:openstack-***@lists.openstack.org>>
Subject: Re: [openstack-dev] [horizon] REST and Django


In your previous example, you are posting to a certain URL (i.e. /keystone/{ver:=x.0}/{method:=update}).
<client: POST /keystone/{ver:=x.0}/{method:=update}> => <middleware: just forward to clients[ver].getattr("method")(**kwargs)> => <keystone: update>

Correct me if I'm wrong, but it looks like you have a unique URL for each /service/version/method.
I fail to see how that is different than what we have today? Is there a view for each service? each version?

Let's say for argument sake that you have a single view that takes care of all URL routing. All requests pass through this view and contain a JSON that contains instruction on which API to invoke and what parameters to pass.
And lets also say that you wrote some code that uses reflection to map the JSON to an action. What you end up with is a client-centric application, where all of the logic resides client-side. If there are things we want to accomplish server-side, it will be extremely hard to pull off. Things like caching, websocket, aggregation, batch actions, translation, etc.... What you end up with is a client with no help from the server.

Obviously the other extreme is what we have today, where we do everything server-side and only using client-side for binding events. I personally prefer a more balance approach where we can leverage both the server and client. There are things that client can do well, and there are things that server can do well. Going the RPC way restrict us to just client technologies and may hamper any additional future functionalities we want to bring server-side. In other words, using REST over RPC gives us the opportunity to use server-side technologies to help solve problems should the need for it arises.

I would also argue that the REST approach is NOT what we have today. What we have today is a static webpage that is generated server-side, where API is hidden from the client. What we end up with using the REST approach is a dynamic webpage generated client-side, two very different things. We have essentially striped out the rendering logic from Django templating and replaced it with Angular.


-----Tihomir Trifonov <***@gmail.com<mailto:***@gmail.com>> wrote: -----
To: "OpenStack Development Mailing List (not for usage questions)" <openstack-***@lists.openstack.org<mailto:openstack-***@lists.openstack.org>>
From: Tihomir Trifonov <***@gmail.com<mailto:***@gmail.com>>
Date: 12/12/2014 04:53AM
Subject: Re: [openstack-dev] [horizon] REST and Django

Here's an example: Admin user Joe has an Domain open and stares at it for 15 minutes while he updates the description. Admin user Bob is asked to go ahead and enable it. He opens the record, edits it, and then saves it. Joe finished perfecting the description and saves it. Doing this action would mean that the Domain is enabled and the description gets updated. Last man in still wins if he updates the same fields, but if they update different fields then both of their changes will take affect without them stomping on each other. Whether that is good or bad may depend on the situation…


That's a great example. I believe that all of the Openstack APIs support PATCH updates of arbitrary fields. This way - the frontend(AngularJS) can detect which fields are being modified, and to submit only these fields for update. If we however use a form with POST, although we will load the object before updating it, the middleware cannot find which fields are actually modified, and will update them all, which is more likely what PUT should do. Thus having full control in the frontend part, we can submit only changed fields. If however a service API doesn't support PATCH, it is actually a problem in the API and not in the client...



The service API documentation almost always lags (although, helped by specs now) and the service team takes on the burden of exposing a programmatic way to access the API. This is tested and easily consumable via the python clients, which removes some guesswork from using the service.

True. But what if the service team modifies a method signature from let's say:

def add_something(self, request,
​ field1, field2):


to

def add_something(self, request,
​ field1, field2, field3):


and in the middleware we have the old signature:

​def add_something(self, request,
​ field1, field2):

we still need to modify the middleware to add the new field. If however the middleware is transparent and just passes **kwargs, it will pass through whatever the frontend sends. So we just need to update the frontend, which can be done using custom views, and not necessary going through an upstream change. My point is why do we need to hide some features of the backend service API behind a "firewall" what the middleware in fact is?







On Fri, Dec 12, 2014 at 8:08 AM, Tripp, Travis S <***@hp.com<mailto:***@hp.com>> wrote:
I just re-read and I apologize for the hastily written email I previously
sent. I’ll try to salvage it with a bit of a revision below (please ignore
the previous email).

On 12/11/14, 7:02 PM, "Tripp, Travis S" <***@hp.com<mailto:***@hp.com>> wrote
(REVISED):

>Tihomir,
>
>Your comments in the patch were very helpful for me to understand your
>concerns about the ease of customizing without requiring upstream
>changes. It also reminded me that I’ve also previously questioned the
>python middleman.
>
>However, here are a couple of bullet points for Devil’s Advocate
>consideration.
>
>
> * Will we take on auto-discovery of API extensions in two spots
>(python for legacy and JS for new)?
> * The Horizon team will have to keep an even closer eye on every
>single project and be ready to react if there are changes to the API that
>break things. Right now in Glance, for example, they are working on some
>fixes to the v2 API (soon to become v2.3) that will allow them to
>deprecate v1 somewhat transparently to users of the client library.
> * The service API documentation almost always lags (although, helped
>by specs now) and the service team takes on the burden of exposing a
>programmatic way to access the API. This is tested and easily consumable
>via the python clients, which removes some guesswork from using the
>service.
> * This is going to be an incremental approach with legacy support
>requirements anyway. So, incorporating python side changes won’t just go
>away.
> * Which approach would be better if we introduce a server side
>caching mechanism or a new source of data such as elastic search to
>improve performance? Would the client side code have to be changed
>dramatically to take advantage of those improvements or could it be done
>transparently on the server side if we own the exposed API?
>
>I’m not sure I fully understood your example about Cinder. Was it the
>cinder client that held up delivery of horizon support, the cinder API or
>both? If the API isn’t in, then it would hold up delivery of the feature
>in any case. There still would be timing pressures to react and build a
>new view that supports it. For customization, with Richard’s approach new
>views could be supported by just dropping in a new REST API decorated
>module with the APIs you want, including direct pass through support if
>desired to new APIs. Downstream customizations / Upstream changes to
>views seem a bit like a bit of a related, but different issue to me as
>long as their is an easy way to drop in new API support.
>
>Finally, regarding the client making two calls to do an update:
>
>​>>Do we really need the lines:​
>
>>> project = api.keystone.tenant_get(request, id)
>>> kwargs = _tenant_kwargs_from_DATA(request.DATA, enabled=None)
>​
>I agree that if you already have all the data it may be bad to have to do
>another call. I do think there is room for discussing the reasoning,
>though.
>As far as I can tell, they do this so that if you are updating an entity,
>you have to be very specific about the fields you are changing. I
>actually see this as potentially a protectionary measure against data
>loss and sometimes a very nice to have feature. It perhaps was intended
>to *help* guard against race conditions (no locking and no transactions
>with many users simultaneously accessing the data).
>
>Here's an example: Admin user Joe has a Domain open and stares at it for
>15 minutes while he updates just the description. Admin user Bob is asked
>to go ahead and enable it. He opens the record, edits it, and then saves
>it. Joe finished perfecting the description and saves it. They could in
>effect both edit the same domain independently. Last man in still wins if
>he updates the same fields, but if they update different fields then both
>of their changes will take affect without them stomping on each other. Or
>maybe it is intended to encourage client users to compare their current
>and previous to see if they should issue a warning if the data changed
>between getting and updating the data. Or maybe like you said, it is just
>overhead API calls.
Tihomir Trifonov
2014-12-15 11:09:44 UTC
Permalink
Travis,

That said, I can see a few ways that we could use the same REST decorator
> code and provide direct access to the API. We’d simply provide a class
> where the url_regex maps to the desired path and gives direct passthrough.
> Maybe that kind of passthrough could always be provided for ease of
> customization / extensibility and additional methods with wrappers provided
> when necessary.



I completely agree on this. We can use the REST decorator to handle either
some really specific cases, or to handle some features, like pagination, in
a general way for all entities, if possible. What I argued against was that
it is unnecessary to have duplicate code JS->REST->APIClient, if we can
call directly JS->(auth wrapper)->APIClient. Also, there are some examples
- where the middleware wrapper hides some functionality, or makes some
unneeded processing, that we may completely skip.

In the given example - we don't need the (images, has_prev, has_more)
return value. What we really need is the list of images, sliced based on
the request.GET parameters and the config.API_RESULT_LIMIT. Then - the
whole check for has_prev, has_more should be done in the client. Currently
this processing was done at the server, as it was needed by the Django
rendering engine. Now it is not. So I am basically talking on
simplification of the middleware layer as much as possible, and moving the
presentation logic into the JS Client.


Also, to answer the comment of Thai - there is a lot of work that the
server will still do - like the translation - I guess we should load the
angular templates from the server with applied translation rather than
putting them into plain js files. I'm not sure what are the best options
here. But still - there is a lot of unneeded code currently in the
openstack_dashboard/api/*.py files.

So I guess the current approach with Django-REST might fit our needs. We
just have to look over each /api/ file in greater detail and to remove the
code that will better work on the client.

Let's move the discussion in Gerrit, and discuss the api wrapper proposed
by Richard. I believe we are on the same page now, I just needed to clarify
for myself that we are not going to just replace the Django with REST, but
we want to make Horizon a really flexible and powerful application.


On Sat, Dec 13, 2014 at 1:09 AM, Tripp, Travis S <***@hp.com>
wrote:
>
> Tihomir,
>
> Today I added one glance call based on Richard’s decorator pattern[1] and
> started to play with incorporating some of your ideas. Please note, I only
> had limited time today. That is passing the kwargs through to the glance
> client. This was an interesting first choice, because it immediately
> highlighted a concrete example of the horizon glance wrapper
> post-processing still being useful (rather than be a direct pass-through
> with no wrapper). See below. If you have some some concrete code examples
> of your ideas it would be helpful.
>
> [1]
> https://review.openstack.org/#/c/141273/2/openstack_dashboard/api/rest/glance.py
>
> With the patch, basically, you can call the following and all of the GET
> parameters get passed directly through to the horizon glance client and you
> get results back as expected.
>
>
> http://localhost:8002/api/glance/images/?sort_dir=desc&sort_key=created_at&paginate=True&marker=bb2cfb1c-2234-4f54-aec5-b4916fe2d747
>
> If you pass in an incorrect sort_key, the glance client returns the
> following error message which propagates back to the REST caller as an
> error with the message:
>
> "sort_key must be one of the following: name, status, container_format,
> disk_format, size, id, created_at, updated_at."
>
> This is done by passing **request.GET.dict() through.
>
> Please note, that if you try this (with POSTMAN, for example), you need to
> set the header of X-Requested-With = XMLHttpRequest
>
> So, what issues did it immediately call out with directly invoking the
> client?
>
> The python-glanceclient internally handles pagination by returning a
> generator. Each iteration on the generator will handle making a request
> for the next page of data. If you were to just do something like return
> list(image_generator) to serialize it back out to the caller, it would
> actually end up making a call back to the server X times to fetch all pages
> before serializing back (thereby not really paginating). The horizon glance
> client wrapper today handles this by using islice intelligently along with
> honoring the API_RESULT_LIMIT setting in Horizon. So, this gives a direct
> example of where the wrapper does something that a direct passthrough to
> the client would not allow.
>
> That said, I can see a few ways that we could use the same REST decorator
> code and provide direct access to the API. We’d simply provide a class
> where the url_regex maps to the desired path and gives direct passthrough.
> Maybe that kind of passthrough could always be provided for ease of
> customization / extensibility and additional methods with wrappers provided
> when necessary. I need to leave for today, so can’t actually try that out
> at the moment.
>
> Thanks,
> Travis
>
> From: Thai Q Tran <***@us.ibm.com<mailto:***@us.ibm.com>>
> Reply-To: OpenStack List <openstack-***@lists.openstack.org<mailto:
> openstack-***@lists.openstack.org>>
> Date: Friday, December 12, 2014 at 11:05 AM
> To: OpenStack List <openstack-***@lists.openstack.org<mailto:
> openstack-***@lists.openstack.org>>
> Subject: Re: [openstack-dev] [horizon] REST and Django
>
>
> In your previous example, you are posting to a certain URL (i.e.
> /keystone/{ver:=x.0}/{method:=update}).
> <client: POST /keystone/{ver:=x.0}/{method:=update}> => <middleware: just
> forward to clients[ver].getattr("method")(**kwargs)> => <keystone: update>
>
> Correct me if I'm wrong, but it looks like you have a unique URL for each
> /service/version/method.
> I fail to see how that is different than what we have today? Is there a
> view for each service? each version?
>
> Let's say for argument sake that you have a single view that takes care of
> all URL routing. All requests pass through this view and contain a JSON
> that contains instruction on which API to invoke and what parameters to
> pass.
> And lets also say that you wrote some code that uses reflection to map the
> JSON to an action. What you end up with is a client-centric application,
> where all of the logic resides client-side. If there are things we want to
> accomplish server-side, it will be extremely hard to pull off. Things like
> caching, websocket, aggregation, batch actions, translation, etc.... What
> you end up with is a client with no help from the server.
>
> Obviously the other extreme is what we have today, where we do everything
> server-side and only using client-side for binding events. I personally
> prefer a more balance approach where we can leverage both the server and
> client. There are things that client can do well, and there are things that
> server can do well. Going the RPC way restrict us to just client
> technologies and may hamper any additional future functionalities we want
> to bring server-side. In other words, using REST over RPC gives us the
> opportunity to use server-side technologies to help solve problems should
> the need for it arises.
>
> I would also argue that the REST approach is NOT what we have today. What
> we have today is a static webpage that is generated server-side, where API
> is hidden from the client. What we end up with using the REST approach is a
> dynamic webpage generated client-side, two very different things. We have
> essentially striped out the rendering logic from Django templating and
> replaced it with Angular.
>
>
> -----Tihomir Trifonov <***@gmail.com<mailto:***@gmail.com>>
> wrote: -----
> To: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-***@lists.openstack.org<mailto:openstack-***@lists.openstack.org
> >>
> From: Tihomir Trifonov <***@gmail.com<mailto:***@gmail.com>>
> Date: 12/12/2014 04:53AM
> Subject: Re: [openstack-dev] [horizon] REST and Django
>
> Here's an example: Admin user Joe has an Domain open and stares at it for
> 15 minutes while he updates the description. Admin user Bob is asked to go
> ahead and enable it. He opens the record, edits it, and then saves it. Joe
> finished perfecting the description and saves it. Doing this action would
> mean that the Domain is enabled and the description gets updated. Last man
> in still wins if he updates the same fields, but if they update different
> fields then both of their changes will take affect without them stomping on
> each other. Whether that is good or bad may depend on the situation

>
>
> That's a great example. I believe that all of the Openstack APIs support
> PATCH updates of arbitrary fields. This way - the frontend(AngularJS) can
> detect which fields are being modified, and to submit only these fields for
> update. If we however use a form with POST, although we will load the
> object before updating it, the middleware cannot find which fields are
> actually modified, and will update them all, which is more likely what PUT
> should do. Thus having full control in the frontend part, we can submit
> only changed fields. If however a service API doesn't support PATCH, it is
> actually a problem in the API and not in the client...
>
>
>
> The service API documentation almost always lags (although, helped by
> specs now) and the service team takes on the burden of exposing a
> programmatic way to access the API. This is tested and easily consumable
> via the python clients, which removes some guesswork from using the service.
>
> True. But what if the service team modifies a method signature from let's
> say:
>
> def add_something(self, request,
> ​ field1, field2):
>
>
> to
>
> def add_something(self, request,
> ​ field1, field2, field3):
>
>
> and in the middleware we have the old signature:
>
> ​def add_something(self, request,
> ​ field1, field2):
>
> we still need to modify the middleware to add the new field. If however
> the middleware is transparent and just passes **kwargs, it will pass
> through whatever the frontend sends. So we just need to update the
> frontend, which can be done using custom views, and not necessary going
> through an upstream change. My point is why do we need to hide some
> features of the backend service API behind a "firewall" what the middleware
> in fact is?
>
>
>
>
>
>
>
> On Fri, Dec 12, 2014 at 8:08 AM, Tripp, Travis S <***@hp.com
> <mailto:***@hp.com>> wrote:
> I just re-read and I apologize for the hastily written email I previously
> sent. I’ll try to salvage it with a bit of a revision below (please ignore
> the previous email).
>
> On 12/11/14, 7:02 PM, "Tripp, Travis S" <***@hp.com<mailto:
> ***@hp.com>> wrote
> (REVISED):
>
> >Tihomir,
> >
> >Your comments in the patch were very helpful for me to understand your
> >concerns about the ease of customizing without requiring upstream
> >changes. It also reminded me that I’ve also previously questioned the
> >python middleman.
> >
> >However, here are a couple of bullet points for Devil’s Advocate
> >consideration.
> >
> >
> > * Will we take on auto-discovery of API extensions in two spots
> >(python for legacy and JS for new)?
> > * The Horizon team will have to keep an even closer eye on every
> >single project and be ready to react if there are changes to the API that
> >break things. Right now in Glance, for example, they are working on some
> >fixes to the v2 API (soon to become v2.3) that will allow them to
> >deprecate v1 somewhat transparently to users of the client library.
> > * The service API documentation almost always lags (although, helped
> >by specs now) and the service team takes on the burden of exposing a
> >programmatic way to access the API. This is tested and easily consumable
> >via the python clients, which removes some guesswork from using the
> >service.
> > * This is going to be an incremental approach with legacy support
> >requirements anyway. So, incorporating python side changes won’t just go
> >away.
> > * Which approach would be better if we introduce a server side
> >caching mechanism or a new source of data such as elastic search to
> >improve performance? Would the client side code have to be changed
> >dramatically to take advantage of those improvements or could it be done
> >transparently on the server side if we own the exposed API?
> >
> >I’m not sure I fully understood your example about Cinder. Was it the
> >cinder client that held up delivery of horizon support, the cinder API or
> >both? If the API isn’t in, then it would hold up delivery of the feature
> >in any case. There still would be timing pressures to react and build a
> >new view that supports it. For customization, with Richard’s approach new
> >views could be supported by just dropping in a new REST API decorated
> >module with the APIs you want, including direct pass through support if
> >desired to new APIs. Downstream customizations / Upstream changes to
> >views seem a bit like a bit of a related, but different issue to me as
> >long as their is an easy way to drop in new API support.
> >
> >Finally, regarding the client making two calls to do an update:
> >
> >​>>Do we really need the lines:​
> >
> >>> project = api.keystone.tenant_get(request, id)
> >>> kwargs = _tenant_kwargs_from_DATA(request.DATA, enabled=None)
> >​
> >I agree that if you already have all the data it may be bad to have to do
> >another call. I do think there is room for discussing the reasoning,
> >though.
> >As far as I can tell, they do this so that if you are updating an entity,
> >you have to be very specific about the fields you are changing. I
> >actually see this as potentially a protectionary measure against data
> >loss and sometimes a very nice to have feature. It perhaps was intended
> >to *help* guard against race conditions (no locking and no transactions
> >with many users simultaneously accessing the data).
> >
> >Here's an example: Admin user Joe has a Domain open and stares at it for
> >15 minutes while he updates just the description. Admin user Bob is asked
> >to go ahead and enable it. He opens the record, edits it, and then saves
> >it. Joe finished perfecting the description and saves it. They could in
> >effect both edit the same domain independently. Last man in still wins if
> >he updates the same fields, but if they update different fields then both
> >of their changes will take affect without them stomping on each other. Or
> >maybe it is intended to encourage client users to compare their current
> >and previous to see if they should issue a warning if the data changed
> >between getting and updating the data. Or maybe like you said, it is just
> >overhead API calls.
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-***@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>


--
Regards,
Tihomir Trifonov
Thai Q Tran
2014-12-10 23:37:12 UTC
Permalink
_______________________________________________
OpenStack-dev mailing list
OpenStack-***@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Richard Jones
2014-12-10 23:44:50 UTC
Permalink
+1 to moving application configuration to the application, out of the
library.


Richard

On Thu Dec 11 2014 at 10:38:20 AM Thai Q Tran <***@us.ibm.com> wrote:

> The way we are structuring our javascripts today is complicated. All of
> our static javascripts reside in /horizon/static and are imported through
> _conf.html and _scripts.html. Notice that there are already some panel
> specific javascripts like: horizon.images.js, horizon.instances.js,
> horizon.users.js. They do not belong in horizon. They belong in
> openstack_dashboard because they are specific to a panel.
>
> Why am I raising this issue now? In Angular, we need controllers written
> in javascript for each panel. As we angularize more and more panels, we
> need to store them in a way that make sense. To me, it make sense for us to
> move _conf and _scripts to openstack_dashboard. Or if this is not possible,
> then provide a mechanism to override them in openstack_dashboard.
>
> Thoughts?
> Thai
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-***@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
Thai Q Tran
2014-12-11 02:20:56 UTC
Permalink
_______________________________________________
OpenStack-dev mailing list
OpenStack-***@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
David Lyle
2014-12-11 17:22:34 UTC
Permalink
I'm probably not understanding the nuance of the question but moving the
_scripts.html file to openstack_dashboard creates some circular
dependencies, does it not? templates/base.html in the horizon side of the
repo includes _scripts.html and insures that the javascript needed by the
existing horizon framework is present.

_conf.html seems like a better candidate for moving as it's more closely
tied to the application code.

David


On Wed, Dec 10, 2014 at 7:20 PM, Thai Q Tran <***@us.ibm.com> wrote:

> Sorry for duplicate mail, forgot the subject.
>
> -----Thai Q Tran/Silicon Valley/IBM wrote: -----
> To: "OpenStack Development Mailing List \(not for usage questions\)" <
> openstack-***@lists.openstack.org>
> From: Thai Q Tran/Silicon Valley/IBM
> Date: 12/10/2014 03:37PM
> Subject: Moving _conf and _scripts to dashboard
>
> The way we are structuring our javascripts today is complicated. All of
> our static javascripts reside in /horizon/static and are imported through
> _conf.html and _scripts.html. Notice that there are already some panel
> specific javascripts like: horizon.images.js, horizon.instances.js,
> horizon.users.js. They do not belong in horizon. They belong in
> openstack_dashboard because they are specific to a panel.
>
> Why am I raising this issue now? In Angular, we need controllers written
> in javascript for each panel. As we angularize more and more panels, we
> need to store them in a way that make sense. To me, it make sense for us to
> move _conf and _scripts to openstack_dashboard. Or if this is not possible,
> then provide a mechanism to override them in openstack_dashboard.
>
> Thoughts?
> Thai
>
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-***@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
Thai Q Tran
2014-12-11 20:22:42 UTC
Permalink
_______________________________________________
OpenStack-dev mailing list
OpenStack-***@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
David Lyle
2014-12-12 17:20:19 UTC
Permalink
Not entirely sure why they both exist either.

So by move, you meant override (nuance). That's different and I have no
issue with that.

I'm also fine with attempting to consolidate _conf and _scripts.

David

On Thu, Dec 11, 2014 at 1:22 PM, Thai Q Tran <***@us.ibm.com> wrote:

>
> It would not create a circular dependency, dashboard would depend on
> horizon - not the latter.
> Scripts that are library specific will live in horizon while scripts that
> are panel specific will live in dashboard.
> Let me draw a more concrete example.
>
> In Horizon
> We know that _script and _conf are included in the base.html
> We create a _script and _conf placeholder file for project overrides
> (similar to _stylesheets and _header)
> In Dashboard
> We create a _script and _conf file with today's content
> It overrides the _script and _conf file in horizon
> Now we can include panel specific scripts without causing circular
> dependency.
>
> In fact, I would like to go further and suggest that _script and _conf be
> combine into a single file.
> Not sure why we need two places to include scripts.
>
>
> -----David Lyle <***@gmail.com> wrote: -----
> To: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-***@lists.openstack.org>
> From: David Lyle <***@gmail.com>
> Date: 12/11/2014 09:23AM
> Subject: Re: [openstack-dev] [Horizon] Moving _conf and _scripts to
> dashboard
>
>
> I'm probably not understanding the nuance of the question but moving the
> _scripts.html file to openstack_dashboard creates some circular
> dependencies, does it not? templates/base.html in the horizon side of the
> repo includes _scripts.html and insures that the javascript needed by the
> existing horizon framework is present.
>
> _conf.html seems like a better candidate for moving as it's more closely
> tied to the application code.
>
> David
>
>
> On Wed, Dec 10, 2014 at 7:20 PM, Thai Q Tran <***@us.ibm.com> wrote:
>
>> Sorry for duplicate mail, forgot the subject.
>>
>> -----Thai Q Tran/Silicon Valley/IBM wrote: -----
>> To: "OpenStack Development Mailing List \(not for usage questions\)" <
>> openstack-***@lists.openstack.org>
>> From: Thai Q Tran/Silicon Valley/IBM
>> Date: 12/10/2014 03:37PM
>> Subject: Moving _conf and _scripts to dashboard
>>
>> The way we are structuring our javascripts today is complicated. All of
>> our static javascripts reside in /horizon/static and are imported through
>> _conf.html and _scripts.html. Notice that there are already some panel
>> specific javascripts like: horizon.images.js, horizon.instances.js,
>> horizon.users.js. They do not belong in horizon. They belong in
>> openstack_dashboard because they are specific to a panel.
>>
>> Why am I raising this issue now? In Angular, we need controllers written
>> in javascript for each panel. As we angularize more and more panels, we
>> need to store them in a way that make sense. To me, it make sense for us to
>> move _conf and _scripts to openstack_dashboard. Or if this is not possible,
>> then provide a mechanism to override them in openstack_dashboard.
>>
>> Thoughts?
>> Thai
>>
>>
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-***@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-***@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-***@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
Lin Hua Cheng
2014-12-12 18:27:29 UTC
Permalink
Consolidating them would break it for users that have customization and
extension on the two templates.

-Lin

On Fri, Dec 12, 2014 at 9:20 AM, David Lyle <***@gmail.com> wrote:
>
> Not entirely sure why they both exist either.
>
> So by move, you meant override (nuance). That's different and I have no
> issue with that.
>
> I'm also fine with attempting to consolidate _conf and _scripts.
>
> David
>
> On Thu, Dec 11, 2014 at 1:22 PM, Thai Q Tran <***@us.ibm.com> wrote:
>
>>
>> It would not create a circular dependency, dashboard would depend on
>> horizon - not the latter.
>> Scripts that are library specific will live in horizon while scripts that
>> are panel specific will live in dashboard.
>> Let me draw a more concrete example.
>>
>> In Horizon
>> We know that _script and _conf are included in the base.html
>> We create a _script and _conf placeholder file for project overrides
>> (similar to _stylesheets and _header)
>> In Dashboard
>> We create a _script and _conf file with today's content
>> It overrides the _script and _conf file in horizon
>> Now we can include panel specific scripts without causing circular
>> dependency.
>>
>> In fact, I would like to go further and suggest that _script and _conf be
>> combine into a single file.
>> Not sure why we need two places to include scripts.
>>
>>
>> -----David Lyle <***@gmail.com> wrote: -----
>> To: "OpenStack Development Mailing List (not for usage questions)" <
>> openstack-***@lists.openstack.org>
>> From: David Lyle <***@gmail.com>
>> Date: 12/11/2014 09:23AM
>> Subject: Re: [openstack-dev] [Horizon] Moving _conf and _scripts to
>> dashboard
>>
>>
>> I'm probably not understanding the nuance of the question but moving the
>> _scripts.html file to openstack_dashboard creates some circular
>> dependencies, does it not? templates/base.html in the horizon side of the
>> repo includes _scripts.html and insures that the javascript needed by the
>> existing horizon framework is present.
>>
>> _conf.html seems like a better candidate for moving as it's more closely
>> tied to the application code.
>>
>> David
>>
>>
>> On Wed, Dec 10, 2014 at 7:20 PM, Thai Q Tran <***@us.ibm.com> wrote:
>>
>>> Sorry for duplicate mail, forgot the subject.
>>>
>>> -----Thai Q Tran/Silicon Valley/IBM wrote: -----
>>> To: "OpenStack Development Mailing List \(not for usage questions\)" <
>>> openstack-***@lists.openstack.org>
>>> From: Thai Q Tran/Silicon Valley/IBM
>>> Date: 12/10/2014 03:37PM
>>> Subject: Moving _conf and _scripts to dashboard
>>>
>>> The way we are structuring our javascripts today is complicated. All of
>>> our static javascripts reside in /horizon/static and are imported through
>>> _conf.html and _scripts.html. Notice that there are already some panel
>>> specific javascripts like: horizon.images.js, horizon.instances.js,
>>> horizon.users.js. They do not belong in horizon. They belong in
>>> openstack_dashboard because they are specific to a panel.
>>>
>>> Why am I raising this issue now? In Angular, we need controllers written
>>> in javascript for each panel. As we angularize more and more panels, we
>>> need to store them in a way that make sense. To me, it make sense for us to
>>> move _conf and _scripts to openstack_dashboard. Or if this is not possible,
>>> then provide a mechanism to override them in openstack_dashboard.
>>>
>>> Thoughts?
>>> Thai
>>>
>>>
>>>
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-***@lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-***@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-***@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-***@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
Thai Q Tran
2014-12-12 18:43:23 UTC
Permalink
_______________________________________________
OpenStack-dev mailing list
OpenStack-***@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Lin Hua Cheng
2014-12-12 23:17:16 UTC
Permalink
Breaking something for existing user is progress, but not forward. :)

I don't mind moving the code around to _scripts file, but simply dropping
the _conf file is my concern since it might already be extended from.
Perhaps document it first that it will be deprecated, and remove it on
later release.

On Fri, Dec 12, 2014 at 10:43 AM, Thai Q Tran <***@us.ibm.com> wrote:
>
> As is the case with anything we change, but that should not stop us from
> making improvements/progress. I would argue that it would make life easier
> for them since all scripts are now in one place.
>
> -----Lin Hua Cheng <***@gmail.com> wrote: -----
> To: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-***@lists.openstack.org>
> From: Lin Hua Cheng <***@gmail.com>
> Date: 12/12/2014 10:28AM
>
> Subject: Re: [openstack-dev] [Horizon] Moving _conf and _scripts to
> dashboard
>
> Consolidating them would break it for users that have customization and
> extension on the two templates.
>
> -Lin
>
> On Fri, Dec 12, 2014 at 9:20 AM, David Lyle <***@gmail.com> wrote:
>>
>> Not entirely sure why they both exist either.
>>
>> So by move, you meant override (nuance). That's different and I have no
>> issue with that.
>>
>> I'm also fine with attempting to consolidate _conf and _scripts.
>>
>> David
>>
>> On Thu, Dec 11, 2014 at 1:22 PM, Thai Q Tran <***@us.ibm.com> wrote:
>>
>>>
>>> It would not create a circular dependency, dashboard would depend on
>>> horizon - not the latter.
>>> Scripts that are library specific will live in horizon while scripts
>>> that are panel specific will live in dashboard.
>>> Let me draw a more concrete example.
>>>
>>> In Horizon
>>> We know that _script and _conf are included in the base.html
>>> We create a _script and _conf placeholder file for project overrides
>>> (similar to _stylesheets and _header)
>>> In Dashboard
>>> We create a _script and _conf file with today's content
>>> It overrides the _script and _conf file in horizon
>>> Now we can include panel specific scripts without causing circular
>>> dependency.
>>>
>>> In fact, I would like to go further and suggest that _script and _conf
>>> be combine into a single file.
>>> Not sure why we need two places to include scripts.
>>>
>>>
>>> -----David Lyle <***@gmail.com> wrote: -----
>>> To: "OpenStack Development Mailing List (not for usage questions)" <
>>> openstack-***@lists.openstack.org>
>>> From: David Lyle <***@gmail.com>
>>> Date: 12/11/2014 09:23AM
>>> Subject: Re: [openstack-dev] [Horizon] Moving _conf and _scripts to
>>> dashboard
>>>
>>>
>>> I'm probably not understanding the nuance of the question but moving the
>>> _scripts.html file to openstack_dashboard creates some circular
>>> dependencies, does it not? templates/base.html in the horizon side of the
>>> repo includes _scripts.html and insures that the javascript needed by the
>>> existing horizon framework is present.
>>>
>>> _conf.html seems like a better candidate for moving as it's more closely
>>> tied to the application code.
>>>
>>> David
>>>
>>>
>>> On Wed, Dec 10, 2014 at 7:20 PM, Thai Q Tran <***@us.ibm.com> wrote:
>>>
>>>> Sorry for duplicate mail, forgot the subject.
>>>>
>>>> -----Thai Q Tran/Silicon Valley/IBM wrote: -----
>>>> To: "OpenStack Development Mailing List \(not for usage questions\)" <
>>>> openstack-***@lists.openstack.org>
>>>> From: Thai Q Tran/Silicon Valley/IBM
>>>> Date: 12/10/2014 03:37PM
>>>> Subject: Moving _conf and _scripts to dashboard
>>>>
>>>> The way we are structuring our javascripts today is complicated. All of
>>>> our static javascripts reside in /horizon/static and are imported through
>>>> _conf.html and _scripts.html. Notice that there are already some panel
>>>> specific javascripts like: horizon.images.js, horizon.instances.js,
>>>> horizon.users.js. They do not belong in horizon. They belong in
>>>> openstack_dashboard because they are specific to a panel.
>>>>
>>>> Why am I raising this issue now? In Angular, we need controllers
>>>> written in javascript for each panel. As we angularize more and more
>>>> panels, we need to store them in a way that make sense. To me, it make
>>>> sense for us to move _conf and _scripts to openstack_dashboard. Or if this
>>>> is not possible, then provide a mechanism to override them in
>>>> openstack_dashboard.
>>>>
>>>> Thoughts?
>>>> Thai
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> OpenStack-dev mailing list
>>>> OpenStack-***@lists.openstack.org
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>>
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-***@lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-***@lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-***@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-***@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-***@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
Thai Q Tran
2014-12-13 00:06:48 UTC
Permalink
_______________________________________________
OpenStack-dev mailing list
OpenStack-***@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Continue reading on narkive:
Loading...