Postgres Pro
Downloads
Home   >   mailing lists

Thread: [pgAdmin4]: Webpacking of static JS/CSS

[pgAdmin4]: Webpacking of static JS/CSS

From
Surinder Kumar
Date:
Hi All,

I and Ashesh come up with following approach;

Divide the JS modules in following components:

​1. ​
Tools
​(tools.js)​ - It will contain all JS modules under tools directory

​2. ​
Browser
​(browser.js)​ - It will contain all JS modules under browser directory

​3. ​
grid.js
 - It will contains ​
Query tool & Datagrid
​ JS modules​
Done

​4. Common.js and/or vendor.js - Common.js will contains all common js files/libraries shared by modules like backform, backbone, underscore etc.

Entry points JS: main.js - It will load above modules ascynchronusly or it will be minified version of all.
define('', ['common', 'tools', 'browser', 'grid'], function(...) {});

or these modules can also be loaded using dependency; For example, load grid.js on database node expand.

Worked so far:

 - Removed CodeMirror out of vendors directory and instead using node_modules/codemirror files. I didn't sent its patch as codemirror CSS is inserted directly into markup.

 - The file grid.js(datagrid.js and sqleditor.js) is generated including all of its dependency, resolved all errors(dependency/JS errors) occurred while running in browser. Modules like explain, file manager, history tab and grid rendering is working except grid tool menu expand not working(will look into this).

 - The file common.js is generated which contains JS files shared among modules.

 - The template JS modules such as node.js, browser.js are not included in webpack since they are templates containing Jinja syntax, they are added as dependency and loaded dynamically. 

As discussed with Ashesh, we need to determine which file is 'static' JS or dynamic JS as we cannot hardcode, till now I didn't find any way to use is_template flag introduced in def get_own_javascripts(self):
​​
 will search how can I use this variable.

​Currently I am working on converting 'requirejs config shim dependency into webpack' using imports-loader and exports-loader which is taking time and will look for alternative if it doesn't works. Once dependency is properly defined, the task to generate other module JS will become quite easy.

I also found, the CSS imported using import 'slickgrid/slick.grid.css'; statement is put into <style></style> tags
​ in html​
, instead must be loaded separately as a file so overrides.css can work.

Thoughts?

Thanks,
Surinder

Re: [pgAdmin4]: Webpacking of static JS/CSS

From
George Gelashvili
Date:
 Hey Surinder,


​1. ​
Tools
​(tools.js)​ - It will contain all JS modules under tools directory


​2. ​
Browser
​(browser.js)​ - It will contain all JS modules under browser directory

By under, do you mean every javascript file recursively under the browser directory? 
 
​4. Common.js and/or vendor.js - Common.js will contains all common js files/libraries shared by modules like backform, backbone, underscore etc.

We're okay with bundling vendorized code for a first pass to webpack everything. The goal should be to pull code out of the shared javascript bundle as we unvendor dependencies.
For naming: Common.js is an overloaded term (http://requirejs.org/docs/commonjs.html). We prefer vendor.js. Will there be non-vendor code put into this bundle?
 
Entry points JS: main.js - It will load above modules ascynchronusly or it will be minified version of all.
define('', ['common', 'tools', 'browser', 'grid'], function(...) {});
or these modules can also be loaded using dependency; For example, load grid.js on database node expand.

Could you go into some more detail on this decision? How would main.js be used in the app? Is the idea to still use define from require.js in javascript being webpacked?

​Currently I am working on converting 'requirejs config shim dependency into webpack' using imports-loader and exports-loader which is taking time and will look for alternative if it doesn't works. Once dependency is properly defined, the task to generate other module JS will become quite easy.

We used imports-loader and exports-loader for tests (see web/webpack.test.config.js)
 
I also found, the CSS imported using import 'slickgrid/slick.grid.css'; statement is put into <style></style> tags
​ in html​
, instead must be loaded separately as a file so overrides.css can work.

Those three css files from slickgrid need to be brought into the application somehow because they won't be available after building the app, since SlickGrid has been un-vendored. We opted to webpack the css together with the required slickgrid js files. Can the overrides.css files be modified to work with this bundle? Which overrides.css file is not working now?
 
Cheers,
Matt and George

Re: [pgAdmin4]: Webpacking of static JS/CSS

From
Surinder Kumar
Date:
Hi

On Fri, Jun 30, 2017 at 2:29 AM, George Gelashvili <ggelashvili@pivotal.io> wrote:
 Hey Surinder,


​1. ​
Tools
​(tools.js)​ - It will contain all JS modules under tools directory


​2. ​
Browser
​(browser.js)​ - It will contain all JS modules under browser directory

By under, do you mean every javascript file recursively under the browser directory? 
​No, I mean a single bundled javascript ​file of various nodes static JS files.
 
​4. Common.js and/or vendor.js - Common.js will contains all common js files/libraries shared by modules like backform, backbone, underscore etc.

We're okay with bundling vendorized code for a first pass to webpack everything. The goal should be to pull code out of the shared javascript bundle as we unvendor dependencies.
Yes, bundled vendor.js will have all unvendor dependencies and files will refer from node_modules.
For naming: Common.js is an overloaded term (http://requirejs.org/docs/commonjs.html). We prefer vendor.js. Will there be non-vendor code put into this bundle?
​I think vendor.js should only contain vendor files. We can create another file(xyz) that will contains JS files shared across the modules.
 
​What do you think ?​
 
​
Entry points JS: main.js - It will load above modules ascynchronusly or it will be minified version of all.
define('', ['common', 'tools', 'browser', 'grid'], function(...) {});
or these modules can also be loaded using dependency; For example, load grid.js on database node expand.

Could you go into some more detail on this decision? How would main.js be used in the app? Is the idea to still use define from require.js in javascript being webpacked?
​In pgAdmin4 there is no main.js, but entry point is browser/index.html.​
This is how I think bundled JS will load:

Load vendor.js using <script></script> in base.html

​browser/​
index.html

require(
​  ​
['pgadmin',
​'sources/generated/common', ​
'sources/generated/bundled'],
​  ​
function() {
}
​)​

sources/generated/bundled
​.js​


​define
(
​  ​
['
​sources/generated/browser
', 'sources/generated/
​tools
'
​, 'sources/generated/grid'​
],
​  ​
function
​(​
pgBrowser) {
​  pgBrowser.init();​
}
)​
​


​or we can use 'import'​ instead of 'require' for files that will be bundled like 'bundled.js'
I keep require/define call untouched in module JS, and webpack converts 'define' calls to __webpack_require__(module_id) to load dependent modules.

​Currently I am working on converting 'requirejs config shim dependency into webpack' using imports-loader and exports-loader which is taking time and will look for alternative if it doesn't works. Once dependency is properly defined, the task to generate other module JS will become quite easy.

We used imports-loader and exports-loader for tests (see web/webpack.test.config.js)
​Hmm. it is useful.​ but I am using shim-loader which is compatible with requirejs shim format.
 
I also found, the CSS imported using import 'slickgrid/slick.grid.css'; statement is put into <style></style> tags
​ in html​
, instead must be loaded separately as a file so overrides.css can work.

Those three css files from slickgrid need to be brought into the application somehow because they won't be available after building the app, since SlickGrid has been un-vendored. We opted to webpack the css together with the required slickgrid js files. Can the overrides.css files be modified to work with this bundle? Which overrides.css file is not working now?
​This affects the CSS font styles of edit grid header or might affect other changes as well. we'll have to look how to override vendor slick grid css which is now rendered in HTML markup.

 
Cheers,
Matt and George

Re: [pgAdmin4]: Webpacking of static JS/CSS

From
Surinder Kumar
Date:
Hi

So far, I have been able to generate bundled 'sources/generated/browser.js' which contains only three browser modules (Server-group, Server, Database) and load it from 'browser/index.html'(entry point) and it works(except few JS error present in file). so will fix them.

Then I will add one by one dashboard module, nodes(cast, schema etc.), tools which i earlier removed(because their JS had conflicts with vendor JS) and make changes accordingly to webpack config and make other modules to load properly.

So far the JS files that won't be the part of  bundled JS and will load separately according to dependency defined in externals in webpack config:

1. browser/utils.js - This file is extracted from browser.js which contains Jinja syntax related code for menus, panels and preference related settings from browser.js
2. endpoints.js
3. messages.js
4. supported_servers.js
5. current_user.js
6. gettext.js
7. translations.js

All other JS files will go into bundled JS.

Currently replacing '{{ url_for }}' in dashboard JS with client side 'url_for' and load with bundled JS.

Thanks,
Surinder

On Fri, Jun 30, 2017 at 3:34 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

On Fri, Jun 30, 2017 at 2:29 AM, George Gelashvili <ggelashvili@pivotal.io> wrote:
 Hey Surinder,


​1. ​
Tools
​(tools.js)​ - It will contain all JS modules under tools directory


​2. ​
Browser
​(browser.js)​ - It will contain all JS modules under browser directory

By under, do you mean every javascript file recursively under the browser directory? 
​No, I mean a single bundled javascript ​file of various nodes static JS files.
 
​4. Common.js and/or vendor.js - Common.js will contains all common js files/libraries shared by modules like backform, backbone, underscore etc.

We're okay with bundling vendorized code for a first pass to webpack everything. The goal should be to pull code out of the shared javascript bundle as we unvendor dependencies.
Yes, bundled vendor.js will have all unvendor dependencies and files will refer from node_modules.
For naming: Common.js is an overloaded term (http://requirejs.org/docs/commonjs.html). We prefer vendor.js. Will there be non-vendor code put into this bundle?
​I think vendor.js should only contain vendor files. We can create another file(xyz) that will contains JS files shared across the modules.
 
​What do you think ?​
 
​
Entry points JS: main.js - It will load above modules ascynchronusly or it will be minified version of all.
define('', ['common', 'tools', 'browser', 'grid'], function(...) {});
or these modules can also be loaded using dependency; For example, load grid.js on database node expand.

Could you go into some more detail on this decision? How would main.js be used in the app? Is the idea to still use define from require.js in javascript being webpacked?
​In pgAdmin4 there is no main.js, but entry point is browser/index.html.​
This is how I think bundled JS will load:

Load vendor.js using <script></script> in base.html

​browser/​
index.html

require(
​  ​
['pgadmin',
​'sources/generated/common', ​
'sources/generated/bundled'],
​  ​
function() {
}
​)​

sources/generated/bundled
​.js​


​define
(
​  ​
['
​sources/generated/browser
', 'sources/generated/
​tools
'
​, 'sources/generated/grid'​
],
​  ​
function
​(​
pgBrowser) {
​  pgBrowser.init();​
}
)​
​


​or we can use 'import'​ instead of 'require' for files that will be bundled like 'bundled.js'
I keep require/define call untouched in module JS, and webpack converts 'define' calls to __webpack_require__(module_id) to load dependent modules.

​Currently I am working on converting 'requirejs config shim dependency into webpack' using imports-loader and exports-loader which is taking time and will look for alternative if it doesn't works. Once dependency is properly defined, the task to generate other module JS will become quite easy.

We used imports-loader and exports-loader for tests (see web/webpack.test.config.js)
​Hmm. it is useful.​ but I am using shim-loader which is compatible with requirejs shim format.
 
I also found, the CSS imported using import 'slickgrid/slick.grid.css'; statement is put into <style></style> tags
​ in html​
, instead must be loaded separately as a file so overrides.css can work.

Those three css files from slickgrid need to be brought into the application somehow because they won't be available after building the app, since SlickGrid has been un-vendored. We opted to webpack the css together with the required slickgrid js files. Can the overrides.css files be modified to work with this bundle? Which overrides.css file is not working now?
​This affects the CSS font styles of edit grid header or might affect other changes as well. we'll have to look how to override vendor slick grid css which is now rendered in HTML markup.

 
Cheers,
Matt and George


Re: [pgAdmin4]: Webpacking of static JS/CSS

From
Dave Page
Date:
Awesome work. Do you have an ETA for a feature complete test patch?

On Fri, Jun 30, 2017 at 11:41 AM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

So far, I have been able to generate bundled 'sources/generated/browser.js' which contains only three browser modules (Server-group, Server, Database) and load it from 'browser/index.html'(entry point) and it works(except few JS error present in file). so will fix them.

Then I will add one by one dashboard module, nodes(cast, schema etc.), tools which i earlier removed(because their JS had conflicts with vendor JS) and make changes accordingly to webpack config and make other modules to load properly.

So far the JS files that won't be the part of  bundled JS and will load separately according to dependency defined in externals in webpack config:

1. browser/utils.js - This file is extracted from browser.js which contains Jinja syntax related code for menus, panels and preference related settings from browser.js
2. endpoints.js
3. messages.js
4. supported_servers.js
5. current_user.js
6. gettext.js
7. translations.js

All other JS files will go into bundled JS.

Currently replacing '{{ url_for }}' in dashboard JS with client side 'url_for' and load with bundled JS.

Thanks,
Surinder

On Fri, Jun 30, 2017 at 3:34 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

On Fri, Jun 30, 2017 at 2:29 AM, George Gelashvili <ggelashvili@pivotal.io> wrote:
 Hey Surinder,


​1. ​
Tools
​(tools.js)​ - It will contain all JS modules under tools directory


​2. ​
Browser
​(browser.js)​ - It will contain all JS modules under browser directory

By under, do you mean every javascript file recursively under the browser directory? 
​No, I mean a single bundled javascript ​file of various nodes static JS files.
 
​4. Common.js and/or vendor.js - Common.js will contains all common js files/libraries shared by modules like backform, backbone, underscore etc.

We're okay with bundling vendorized code for a first pass to webpack everything. The goal should be to pull code out of the shared javascript bundle as we unvendor dependencies.
Yes, bundled vendor.js will have all unvendor dependencies and files will refer from node_modules.
For naming: Common.js is an overloaded term (http://requirejs.org/docs/commonjs.html). We prefer vendor.js. Will there be non-vendor code put into this bundle?
​I think vendor.js should only contain vendor files. We can create another file(xyz) that will contains JS files shared across the modules.
 
​What do you think ?​
 
​
Entry points JS: main.js - It will load above modules ascynchronusly or it will be minified version of all.
define('', ['common', 'tools', 'browser', 'grid'], function(...) {});
or these modules can also be loaded using dependency; For example, load grid.js on database node expand.

Could you go into some more detail on this decision? How would main.js be used in the app? Is the idea to still use define from require.js in javascript being webpacked?
​In pgAdmin4 there is no main.js, but entry point is browser/index.html.​
This is how I think bundled JS will load:

Load vendor.js using <script></script> in base.html

​browser/​
index.html

require(
​  ​
['pgadmin',
​'sources/generated/common', ​
'sources/generated/bundled'],
​  ​
function() {
}
​)​

sources/generated/bundled
​.js​


​define
(
​  ​
['
​sources/generated/browser
', 'sources/generated/
​tools
'
​, 'sources/generated/grid'​
],
​  ​
function
​(​
pgBrowser) {
​  pgBrowser.init();​
}
)​
​


​or we can use 'import'​ instead of 'require' for files that will be bundled like 'bundled.js'
I keep require/define call untouched in module JS, and webpack converts 'define' calls to __webpack_require__(module_id) to load dependent modules.

​Currently I am working on converting 'requirejs config shim dependency into webpack' using imports-loader and exports-loader which is taking time and will look for alternative if it doesn't works. Once dependency is properly defined, the task to generate other module JS will become quite easy.

We used imports-loader and exports-loader for tests (see web/webpack.test.config.js)
​Hmm. it is useful.​ but I am using shim-loader which is compatible with requirejs shim format.
 
I also found, the CSS imported using import 'slickgrid/slick.grid.css'; statement is put into <style></style> tags
​ in html​
, instead must be loaded separately as a file so overrides.css can work.

Those three css files from slickgrid need to be brought into the application somehow because they won't be available after building the app, since SlickGrid has been un-vendored. We opted to webpack the css together with the required slickgrid js files. Can the overrides.css files be modified to work with this bundle? Which overrides.css file is not working now?
​This affects the CSS font styles of edit grid header or might affect other changes as well. we'll have to look how to override vendor slick grid css which is now rendered in HTML markup.

 
Cheers,
Matt and George





--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Re: [pgAdmin4]: Webpacking of static JS/CSS

From
Surinder Kumar
Date:
Hi All,

​Things to discuss:

How to differentiate between a static and template JS
​​
.

​In Webpack we have to resolve paths to the modules same as in requirejs.config(base.html). but previously we were using 'current_app.javascripts' which holds the the path references for modules registered. Now in webpack.config.js we don't have such variable. so the path to module js has to give manually.

However, there are ​t
wo ways:

​1. ​
The convention should be that static files should go in ‘static/‘ dir and templates in ‘templates/‘ dir.
​ If this convention is followed, we can search through directories and pull out static and template file paths.​

​2. ​
As per Ashesh, 
​'​
static/
'​
 dir may also have templates file
​ or vice versa​
​.​
 
​I​
n this case, the convention to identify template file 
​can be
, 
​templates ​
file must ends with ‘.template.js’ extension
​.
​
The flag ‘is_template’ added in every module’s __init__.py doesn’t help here because webpack works on files and directories.
​ so there is no way to use this flag.​

For debug/release mode:

We’ll get rid of snippets in 
​'​
get_own_javascripts
​'​
 and 
​'​
get_own_stylesheets
​'​
 defined in modules __init__.py as they are
​ of​
 no use for now as CSS and JS are
​ going to​
 bundl
​e​
. They 
​were
 used to load specified CSS and JS files
​ at runtime.​

Introduce a flag ‘-p’ in 
​'​
yarn run bundle
​ -p'
 command to tell webpack config to run in development or production mode. -p is for production.

Few 
​W​
ebpack plugins such as UglifyPlugin or other plugins will run only in production mode to minify JS otherwise load the generated bundle JS(unminified) in debug mode to help with debugging code.

Taks remaining:

​1. ​
Fix local variables which are declared without using var, have to check in each file
​ by​
 running eslint (For now, i will fix only errors which are giving error in browser).

​2. ​
Move non-template files from ’templates’ to ’static’ directory. List of
​ pending​
 modules is here: 
  • Tools (mostly all modules - 9 modules) 
  • Browser nodes - 3 modules(resource group, roles, tablespace)
  • ​About
    ​​
Also can we move 
​'​
dashboard, statistic
​s​
, preferences and help
​'​
 modules inside misc to preserve modularity as pgAdmin is modular
​ ?​

​3. Client side implementation of 'url_for' for several modules.

​4. ​
Integrating CSS bundling patch, also pull out inline CSS
​ and unvendor the CSS.​
​Today i worked on it but it is not completed yet, has some effect on UI, i am looking into it.​

​5. ​
FileManager module is not working
​ 
as it is loaded via html file
​ and the modules it calls are unavailable at that time. 
​I will​
 look into it
​.​
​​

​There are other minor issues that has to take care.​

​6. ​
Caching:

I haven’t read much in detail on caching, but it can used to help browser know when to cache static resources and when to not. Basically, it give names to generated bundled files with unique hash and that hash is changed when a new change in file is detected on running ‘yarn run bundle’.
Also we have to use HTML Webpack plugin to regenerate 
​'​
base.html
​' 
again with new hash in bundled files
​.


​Thanks,
Surinder​

On Fri, Jun 30, 2017 at 4:40 PM, Dave Page <dpage@pgadmin.org> wrote:
Awesome work. Do you have an ETA for a feature complete test patch?

On Fri, Jun 30, 2017 at 11:41 AM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

So far, I have been able to generate bundled 'sources/generated/browser.js' which contains only three browser modules (Server-group, Server, Database) and load it from 'browser/index.html'(entry point) and it works(except few JS error present in file). so will fix them.

Then I will add one by one dashboard module, nodes(cast, schema etc.), tools which i earlier removed(because their JS had conflicts with vendor JS) and make changes accordingly to webpack config and make other modules to load properly.

So far the JS files that won't be the part of  bundled JS and will load separately according to dependency defined in externals in webpack config:

1. browser/utils.js - This file is extracted from browser.js which contains Jinja syntax related code for menus, panels and preference related settings from browser.js
2. endpoints.js
3. messages.js
4. supported_servers.js
5. current_user.js
6. gettext.js
7. translations.js

All other JS files will go into bundled JS.

Currently replacing '{{ url_for }}' in dashboard JS with client side 'url_for' and load with bundled JS.

Thanks,
Surinder

On Fri, Jun 30, 2017 at 3:34 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

On Fri, Jun 30, 2017 at 2:29 AM, George Gelashvili <ggelashvili@pivotal.io> wrote:
 Hey Surinder,


​1. ​
Tools
​(tools.js)​ - It will contain all JS modules under tools directory


​2. ​
Browser
​(browser.js)​ - It will contain all JS modules under browser directory

By under, do you mean every javascript file recursively under the browser directory? 
​No, I mean a single bundled javascript ​file of various nodes static JS files.
 
​4. Common.js and/or vendor.js - Common.js will contains all common js files/libraries shared by modules like backform, backbone, underscore etc.

We're okay with bundling vendorized code for a first pass to webpack everything. The goal should be to pull code out of the shared javascript bundle as we unvendor dependencies.
Yes, bundled vendor.js will have all unvendor dependencies and files will refer from node_modules.
For naming: Common.js is an overloaded term (http://requirejs.org/docs/commonjs.html). We prefer vendor.js. Will there be non-vendor code put into this bundle?
​I think vendor.js should only contain vendor files. We can create another file(xyz) that will contains JS files shared across the modules.
 
​What do you think ?​
 
​
Entry points JS: main.js - It will load above modules ascynchronusly or it will be minified version of all.
define('', ['common', 'tools', 'browser', 'grid'], function(...) {});
or these modules can also be loaded using dependency; For example, load grid.js on database node expand.

Could you go into some more detail on this decision? How would main.js be used in the app? Is the idea to still use define from require.js in javascript being webpacked?
​In pgAdmin4 there is no main.js, but entry point is browser/index.html.​
This is how I think bundled JS will load:

Load vendor.js using <script></script> in base.html

​browser/​
index.html

require(
​  ​
['pgadmin',
​'sources/generated/common', ​
'sources/generated/bundled'],
​  ​
function() {
}
​)​

sources/generated/bundled
​.js​


​define
(
​  ​
['
​sources/generated/browser
', 'sources/generated/
​tools
'
​, 'sources/generated/grid'​
],
​  ​
function
​(​
pgBrowser) {
​  pgBrowser.init();​
}
)​
​


​or we can use 'import'​ instead of 'require' for files that will be bundled like 'bundled.js'
I keep require/define call untouched in module JS, and webpack converts 'define' calls to __webpack_require__(module_id) to load dependent modules.

​Currently I am working on converting 'requirejs config shim dependency into webpack' using imports-loader and exports-loader which is taking time and will look for alternative if it doesn't works. Once dependency is properly defined, the task to generate other module JS will become quite easy.

We used imports-loader and exports-loader for tests (see web/webpack.test.config.js)
​Hmm. it is useful.​ but I am using shim-loader which is compatible with requirejs shim format.
 
I also found, the CSS imported using import 'slickgrid/slick.grid.css'; statement is put into <style></style> tags
​ in html​
, instead must be loaded separately as a file so overrides.css can work.

Those three css files from slickgrid need to be brought into the application somehow because they won't be available after building the app, since SlickGrid has been un-vendored. We opted to webpack the css together with the required slickgrid js files. Can the overrides.css files be modified to work with this bundle? Which overrides.css file is not working now?
​This affects the CSS font styles of edit grid header or might affect other changes as well. we'll have to look how to override vendor slick grid css which is now rendered in HTML markup.

 
Cheers,
Matt and George





--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Re: [pgAdmin4]: Webpacking of static JS/CSS

From
Sarah McAlear
Date:
Hello,


​Things to discuss:

How to differentiate between a static and template JS
​​
.

What is the advantage of webpacking templated JS? It seems as though this creates a system in which the bundled dependencies have to refer back to the backend to load the templates. 

If there is a performance win in packing templated JS then looking at it makes sense.  Otherwise it may make sense to put off until it is clear that the templated files should be dealt with by either de-templating them or bundling them where there is a clear reason.

However, we're wondering about possible performance penalties with templating larger files (as opposed to templating on-demand.) Since jinja templates can execute arbitrary python, this could get time expensive and further slow things like initial page-load.
Another concern is: what happens when a template gets out of date (e.g. if browser.js had previously filled in the content for 'panel_item.content' and had been cached, would it render a new version with the new values when needed? Or is it possible that we would get old content?) 
 
Taks remaining:

​1. ​
Fix local variables which are declared without using var, have to check in each file
​ by​
 running eslint (For now, i will fix only errors which are giving error in browser).

​2. ​
Move non-template files from ’templates’ to ’static’ directory. List of
​ pending​
 modules is here: 
  • Tools (mostly all modules - 9 modules) 
  • Browser nodes - 3 modules(resource group, roles, tablespace)
  • ​About
    ​​
Also can we move 
​'​
dashboard, statistic
​s​
, preferences and help
​'​
 modules inside misc to preserve modularity as pgAdmin is modular
​ ?​

Is there anything from a organization stance you discussed in the previous email that needs to be done to make this usable and consistent?


Thanks,

George & Sarah 

Re: [pgAdmin4]: Webpacking of static JS/CSS

From
Dave Page
Date:
When you say "un-vendorising", do you mean removing them from the vendor directory and adding them to packages.json so they're pulled in via npm/yarn? (if so, good :-) )

On Tue, Jul 11, 2017 at 7:34 AM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

Detailed changes:

​1) Created a file bundle/app.js which loads `browser.js`  and then 'tools.datagrid' and its other dependents are configured to load in '`imports-loader`

2) bundle/codemirror.js - it generates a bundled JS which is used wherever required in various modules.

3) file_utils.js - It bundles the file manager's utility JS file and loaded from `file manager/index.html`

4) lib_css - It contains list of vendor CSS to be bundled.

5) pgadmin_css - It contains list of overrides CSS which are bundled and which has to load after vendors CSS is loaded.

Various Webpack configuration parameters:

1) externals: List of template files to be loaded dynamically when a call is made by dependent modules. These files are excluded from the bundled JS.

2) resolve > alias - This resolves the path of module JS which is referred in other modules; For example: 
'backbone': path.join(__dirname, './node_modules/backbone/backbone')

3) `backbone` is used in 'define(['backbone'])' calls and its path is resolved to above absolute path.

4) 'pgLibs': List of files to bundle into `pgadmin_commons.js`. The purpose is to separate files other than browser node modules JS in `pgadmin_commons.js` and browser node modules JS in `app.bundle.js`. It will help in debugging the code during development.

Un-vendor modules:

All modules are un-vendored except:
- requireJS
- Backgrid JS - it gives reference error when importing backgrid.css from node_modules in bundle/lib.css even if the path is correct. I logged an issue:
Opened an issue

- React JS - I didn't un-vendor React JS because the pivotal developers submitted a patch to fix issue of QtWebEngine. Once the patch is merged in React repo, we will un-vendor.

Other issues faced:

1) Backbone JS: I didn't upgraded backbone JS to latest because it affects the application code mainly `Preferences module`. 

2) jQuery: I didn't upgraded it to latest because it is gives error in loading wcIframe of wcDocker in Query tool. I submit a PR.

3) Acitree - it is not available in yarn repository. logged an request
However i am managing it on my github account by forking this repo.

Specified the repo github link to `acitree` in package.json with tag to pull from 
4.5.0-rc.7. The latest version of actiree has issues so code is used form 4.5.0-rc.7 tag. 

Thanks to bestander for helping this out. link to thread

Plugins used:

​- optimize-css-assets-webpack-plugin: its job is to optimise the CSS bundle like minifying CSS and removing comments from it. [used only in production]

-  uglifyJS - It minimise the bundled JS​, and remove console statements from code. [used only in production]

- definePlugin - It helps in minimising the `React' production bundle. As react has conditional code based on 'NODE_ENV' variable. [used only in production]

- providePlugin - It makes the variable defined through out the application context. For example: jQuery object can be accessed wherever  it is used but not included in `define` calls.

- CommonsChunkPlugin - it helps in separating vendor like code, common dependent modules used in multiple modules. it extracts out in a file.

- extractTextPlugin - it is used in combination with 'css-loader' and 'style-loader'. It helps in extracting CSS and moved into files.

- webpack-bundle-analyzer - it helps in analysing the bundled JS files like size of bundled JS and which JS files are included in bundle. It is commented out. [Use only when need to analyse]

Loaders used:

​- shim-loader: It manages the module dependency, uses the same configuration used in ​requireJS

- imports-loader: It also used to loaded dependent modules which are defined in its `use` setting.

- file-loader - It helps in extracting font, image files into a separate directory.

- css-loader - Imports css files included in modules using `require` and `import` calls.

TODO:

​1) Automatically handle static and template JS files: This is already being discussed. Once it is sorted out, we will change webpack configuration accordingly.

2) Implementing Caching: I will look into this once an initial patch is commited. and later on add as improvement.

3) Source maps: It will help in debugging bundled JS code in production environment.

4) Feature tests are failing: I am currently looking into it. Query tool functionality is working fine, the issue is it is unable to find code mirror.

Steps to run:

​After applying patch: git apply --binary /path/to/patch

run `yarn install`
then run:

​In development mode:
yarn run bundle:dev

In production mode:
yarn run bundle:prod​​

Performance comparison:

On Mac's Chrome - Before bundle it was taking ~9sec to load page. After bundle it took 3.5secs (with no cache)

Please review the patch.

Thanks,
Surinder


On Wed, Jul 5, 2017 at 8:22 PM, Sarah McAlear <smcalear@pivotal.io> wrote:
Hello,


​Things to discuss:

How to differentiate between a static and template JS
​​
.

What is the advantage of webpacking templated JS? It seems as though this creates a system in which the bundled dependencies have to refer back to the backend to load the templates. 

If there is a performance win in packing templated JS then looking at it makes sense.  Otherwise it may make sense to put off until it is clear that the templated files should be dealt with by either de-templating them or bundling them where there is a clear reason.

However, we're wondering about possible performance penalties with templating larger files (as opposed to templating on-demand.) Since jinja templates can execute arbitrary python, this could get time expensive and further slow things like initial page-load.
Another concern is: what happens when a template gets out of date (e.g. if browser.js had previously filled in the content for 'panel_item.content' and had been cached, would it render a new version with the new values when needed? Or is it possible that we would get old content?) 
 
Taks remaining:

​1. ​
Fix local variables which are declared without using var, have to check in each file
​ by​
 running eslint (For now, i will fix only errors which are giving error in browser).

​2. ​
Move non-template files from ’templates’ to ’static’ directory. List of
​ pending​
 modules is here: 
  • Tools (mostly all modules - 9 modules) 
  • Browser nodes - 3 modules(resource group, roles, tablespace)
  • ​About
    ​​
Also can we move 
​'​
dashboard, statistic
​s​
, preferences and help
​'​
 modules inside misc to preserve modularity as pgAdmin is modular
​ ?​

Is there anything from a organization stance you discussed in the previous email that needs to be done to make this usable and consistent?


Thanks,

George & Sarah 




--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Re: [pgAdmin4]: Webpacking of static JS/CSS

From
Surinder Kumar
Date:

On Tue, Jul 11, 2017 at 1:22 PM, Dave Page <dpage@pgadmin.org> wrote:
When you say "un-vendorising", do you mean removing them from the vendor directory and adding them to packages.json so they're pulled in via npm/yarn? (if so, good :-) )
​Yes.​

On Tue, Jul 11, 2017 at 7:34 AM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

Detailed changes:

​1) Created a file bundle/app.js which loads `browser.js`  and then 'tools.datagrid' and its other dependents are configured to load in '`imports-loader`

2) bundle/codemirror.js - it generates a bundled JS which is used wherever required in various modules.

3) file_utils.js - It bundles the file manager's utility JS file and loaded from `file manager/index.html`

4) lib_css - It contains list of vendor CSS to be bundled.

5) pgadmin_css - It contains list of overrides CSS which are bundled and which has to load after vendors CSS is loaded.

Various Webpack configuration parameters:

1) externals: List of template files to be loaded dynamically when a call is made by dependent modules. These files are excluded from the bundled JS.

2) resolve > alias - This resolves the path of module JS which is referred in other modules; For example: 
'backbone': path.join(__dirname, './node_modules/backbone/backbone')

3) `backbone` is used in 'define(['backbone'])' calls and its path is resolved to above absolute path.

4) 'pgLibs': List of files to bundle into `pgadmin_commons.js`. The purpose is to separate files other than browser node modules JS in `pgadmin_commons.js` and browser node modules JS in `app.bundle.js`. It will help in debugging the code during development.

Un-vendor modules:

All modules are un-vendored except:
- requireJS
- Backgrid JS - it gives reference error when importing backgrid.css from node_modules in bundle/lib.css even if the path is correct. I logged an issue:
Opened an issue

- React JS - I didn't un-vendor React JS because the pivotal developers submitted a patch to fix issue of QtWebEngine. Once the patch is merged in React repo, we will un-vendor.

Other issues faced:

1) Backbone JS: I didn't upgraded backbone JS to latest because it affects the application code mainly `Preferences module`. 

2) jQuery: I didn't upgraded it to latest because it is gives error in loading wcIframe of wcDocker in Query tool. I submit a PR.

3) Acitree - it is not available in yarn repository. logged an request
However i am managing it on my github account by forking this repo.

Specified the repo github link to `acitree` in package.json with tag to pull from 
4.5.0-rc.7. The latest version of actiree has issues so code is used form 4.5.0-rc.7 tag. 

Thanks to bestander for helping this out. link to thread

Plugins used:

​- optimize-css-assets-webpack-plugin: its job is to optimise the CSS bundle like minifying CSS and removing comments from it. [used only in production]

-  uglifyJS - It minimise the bundled JS​, and remove console statements from code. [used only in production]

- definePlugin - It helps in minimising the `React' production bundle. As react has conditional code based on 'NODE_ENV' variable. [used only in production]

- providePlugin - It makes the variable defined through out the application context. For example: jQuery object can be accessed wherever  it is used but not included in `define` calls.

- CommonsChunkPlugin - it helps in separating vendor like code, common dependent modules used in multiple modules. it extracts out in a file.

- extractTextPlugin - it is used in combination with 'css-loader' and 'style-loader'. It helps in extracting CSS and moved into files.

- webpack-bundle-analyzer - it helps in analysing the bundled JS files like size of bundled JS and which JS files are included in bundle. It is commented out. [Use only when need to analyse]

Loaders used:

​- shim-loader: It manages the module dependency, uses the same configuration used in ​requireJS

- imports-loader: It also used to loaded dependent modules which are defined in its `use` setting.

- file-loader - It helps in extracting font, image files into a separate directory.

- css-loader - Imports css files included in modules using `require` and `import` calls.

TODO:

​1) Automatically handle static and template JS files: This is already being discussed. Once it is sorted out, we will change webpack configuration accordingly.

2) Implementing Caching: I will look into this once an initial patch is commited. and later on add as improvement.

3) Source maps: It will help in debugging bundled JS code in production environment.

4) Feature tests are failing: I am currently looking into it. Query tool functionality is working fine, the issue is it is unable to find code mirror.

Steps to run:

​After applying patch: git apply --binary /path/to/patch

run `yarn install`
then run:

​In development mode:
yarn run bundle:dev

In production mode:
yarn run bundle:prod​​

Performance comparison:

On Mac's Chrome - Before bundle it was taking ~9sec to load page. After bundle it took 3.5secs (with no cache)

Please review the patch.

Thanks,
Surinder


On Wed, Jul 5, 2017 at 8:22 PM, Sarah McAlear <smcalear@pivotal.io> wrote:
Hello,


​Things to discuss:

How to differentiate between a static and template JS
​​
.

What is the advantage of webpacking templated JS? It seems as though this creates a system in which the bundled dependencies have to refer back to the backend to load the templates. 

If there is a performance win in packing templated JS then looking at it makes sense.  Otherwise it may make sense to put off until it is clear that the templated files should be dealt with by either de-templating them or bundling them where there is a clear reason.

However, we're wondering about possible performance penalties with templating larger files (as opposed to templating on-demand.) Since jinja templates can execute arbitrary python, this could get time expensive and further slow things like initial page-load.
Another concern is: what happens when a template gets out of date (e.g. if browser.js had previously filled in the content for 'panel_item.content' and had been cached, would it render a new version with the new values when needed? Or is it possible that we would get old content?) 
 
Taks remaining:

​1. ​
Fix local variables which are declared without using var, have to check in each file
​ by​
 running eslint (For now, i will fix only errors which are giving error in browser).

​2. ​
Move non-template files from ’templates’ to ’static’ directory. List of
​ pending​
 modules is here: 
  • Tools (mostly all modules - 9 modules) 
  • Browser nodes - 3 modules(resource group, roles, tablespace)
  • ​About
    ​​
Also can we move 
​'​
dashboard, statistic
​s​
, preferences and help
​'​
 modules inside misc to preserve modularity as pgAdmin is modular
​ ?​

Is there anything from a organization stance you discussed in the previous email that needs to be done to make this usable and consistent?


Thanks,

George & Sarah 




--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Re: [pgAdmin4]: Webpacking of static JS/CSS

From
Surinder Kumar
Date:
Hi 

I found this patch won't work on IE, so i have fixed it and will send updated patch.

On Tue, Jul 11, 2017 at 1:25 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:

On Tue, Jul 11, 2017 at 1:22 PM, Dave Page <dpage@pgadmin.org> wrote:
When you say "un-vendorising", do you mean removing them from the vendor directory and adding them to packages.json so they're pulled in via npm/yarn? (if so, good :-) )
​Yes.​

On Tue, Jul 11, 2017 at 7:34 AM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

Detailed changes:

​1) Created a file bundle/app.js which loads `browser.js`  and then 'tools.datagrid' and its other dependents are configured to load in '`imports-loader`

2) bundle/codemirror.js - it generates a bundled JS which is used wherever required in various modules.

3) file_utils.js - It bundles the file manager's utility JS file and loaded from `file manager/index.html`

4) lib_css - It contains list of vendor CSS to be bundled.

5) pgadmin_css - It contains list of overrides CSS which are bundled and which has to load after vendors CSS is loaded.

Various Webpack configuration parameters:

1) externals: List of template files to be loaded dynamically when a call is made by dependent modules. These files are excluded from the bundled JS.

2) resolve > alias - This resolves the path of module JS which is referred in other modules; For example: 
'backbone': path.join(__dirname, './node_modules/backbone/backbone')

3) `backbone` is used in 'define(['backbone'])' calls and its path is resolved to above absolute path.

4) 'pgLibs': List of files to bundle into `pgadmin_commons.js`. The purpose is to separate files other than browser node modules JS in `pgadmin_commons.js` and browser node modules JS in `app.bundle.js`. It will help in debugging the code during development.

Un-vendor modules:

All modules are un-vendored except:
- requireJS
- Backgrid JS - it gives reference error when importing backgrid.css from node_modules in bundle/lib.css even if the path is correct. I logged an issue:
Opened an issue

- React JS - I didn't un-vendor React JS because the pivotal developers submitted a patch to fix issue of QtWebEngine. Once the patch is merged in React repo, we will un-vendor.

Other issues faced:

1) Backbone JS: I didn't upgraded backbone JS to latest because it affects the application code mainly `Preferences module`. 

2) jQuery: I didn't upgraded it to latest because it is gives error in loading wcIframe of wcDocker in Query tool. I submit a PR.

3) Acitree - it is not available in yarn repository. logged an request
However i am managing it on my github account by forking this repo.

Specified the repo github link to `acitree` in package.json with tag to pull from 
4.5.0-rc.7. The latest version of actiree has issues so code is used form 4.5.0-rc.7 tag. 

Thanks to bestander for helping this out. link to thread

Plugins used:

​- optimize-css-assets-webpack-plugin: its job is to optimise the CSS bundle like minifying CSS and removing comments from it. [used only in production]

-  uglifyJS - It minimise the bundled JS​, and remove console statements from code. [used only in production]

- definePlugin - It helps in minimising the `React' production bundle. As react has conditional code based on 'NODE_ENV' variable. [used only in production]

- providePlugin - It makes the variable defined through out the application context. For example: jQuery object can be accessed wherever  it is used but not included in `define` calls.

- CommonsChunkPlugin - it helps in separating vendor like code, common dependent modules used in multiple modules. it extracts out in a file.

- extractTextPlugin - it is used in combination with 'css-loader' and 'style-loader'. It helps in extracting CSS and moved into files.

- webpack-bundle-analyzer - it helps in analysing the bundled JS files like size of bundled JS and which JS files are included in bundle. It is commented out. [Use only when need to analyse]

Loaders used:

​- shim-loader: It manages the module dependency, uses the same configuration used in ​requireJS

- imports-loader: It also used to loaded dependent modules which are defined in its `use` setting.

- file-loader - It helps in extracting font, image files into a separate directory.

- css-loader - Imports css files included in modules using `require` and `import` calls.

TODO:

​1) Automatically handle static and template JS files: This is already being discussed. Once it is sorted out, we will change webpack configuration accordingly.

2) Implementing Caching: I will look into this once an initial patch is commited. and later on add as improvement.

3) Source maps: It will help in debugging bundled JS code in production environment.

4) Feature tests are failing: I am currently looking into it. Query tool functionality is working fine, the issue is it is unable to find code mirror.

Steps to run:

​After applying patch: git apply --binary /path/to/patch

run `yarn install`
then run:

​In development mode:
yarn run bundle:dev

In production mode:
yarn run bundle:prod​​

Performance comparison:

On Mac's Chrome - Before bundle it was taking ~9sec to load page. After bundle it took 3.5secs (with no cache)

Please review the patch.

Thanks,
Surinder


On Wed, Jul 5, 2017 at 8:22 PM, Sarah McAlear <smcalear@pivotal.io> wrote:
Hello,


​Things to discuss:

How to differentiate between a static and template JS
​​
.

What is the advantage of webpacking templated JS? It seems as though this creates a system in which the bundled dependencies have to refer back to the backend to load the templates. 

If there is a performance win in packing templated JS then looking at it makes sense.  Otherwise it may make sense to put off until it is clear that the templated files should be dealt with by either de-templating them or bundling them where there is a clear reason.

However, we're wondering about possible performance penalties with templating larger files (as opposed to templating on-demand.) Since jinja templates can execute arbitrary python, this could get time expensive and further slow things like initial page-load.
Another concern is: what happens when a template gets out of date (e.g. if browser.js had previously filled in the content for 'panel_item.content' and had been cached, would it render a new version with the new values when needed? Or is it possible that we would get old content?) 
 
Taks remaining:

​1. ​
Fix local variables which are declared without using var, have to check in each file
​ by​
 running eslint (For now, i will fix only errors which are giving error in browser).

​2. ​
Move non-template files from ’templates’ to ’static’ directory. List of
​ pending​
 modules is here: 
  • Tools (mostly all modules - 9 modules) 
  • Browser nodes - 3 modules(resource group, roles, tablespace)
  • ​About
    ​​
Also can we move 
​'​
dashboard, statistic
​s​
, preferences and help
​'​
 modules inside misc to preserve modularity as pgAdmin is modular
​ ?​

Is there anything from a organization stance you discussed in the previous email that needs to be done to make this usable and consistent?


Thanks,

George & Sarah 




--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


Re: [pgAdmin4]: Webpacking of static JS/CSS

From
Neel Patel
Date:
Hi Dave,

I have tested Surinder's patch in my local machine with Qt 5.9.1 Webkit on Windows and we are getting improvements with this patch. :)

Below performance tested with Qt 5.9.1 with annulen webkit v 5.212.0 Alpha2.

Before Webpack patch:-

It took ~20 seconds. This 20 seconds includes when user double click on application ( timing of python server start + page load )

After Webpack patch:-

It took ~13 seconds ( timing of python server start + page load ).

We got ~7 seconds improvement with webpack on same machine & same Qt configuration and this will be useful in windows performance issue as well :)

Thanks,
Neel  Patel

On Tue, Jul 11, 2017 at 1:27 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi 

I found this patch won't work on IE, so i have fixed it and will send updated patch.

On Tue, Jul 11, 2017 at 1:25 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:

On Tue, Jul 11, 2017 at 1:22 PM, Dave Page <dpage@pgadmin.org> wrote:
When you say "un-vendorising", do you mean removing them from the vendor directory and adding them to packages.json so they're pulled in via npm/yarn? (if so, good :-) )
​Yes.​

On Tue, Jul 11, 2017 at 7:34 AM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

Detailed changes:

​1) Created a file bundle/app.js which loads `browser.js`  and then 'tools.datagrid' and its other dependents are configured to load in '`imports-loader`

2) bundle/codemirror.js - it generates a bundled JS which is used wherever required in various modules.

3) file_utils.js - It bundles the file manager's utility JS file and loaded from `file manager/index.html`

4) lib_css - It contains list of vendor CSS to be bundled.

5) pgadmin_css - It contains list of overrides CSS which are bundled and which has to load after vendors CSS is loaded.

Various Webpack configuration parameters:

1) externals: List of template files to be loaded dynamically when a call is made by dependent modules. These files are excluded from the bundled JS.

2) resolve > alias - This resolves the path of module JS which is referred in other modules; For example: 
'backbone': path.join(__dirname, './node_modules/backbone/backbone')

3) `backbone` is used in 'define(['backbone'])' calls and its path is resolved to above absolute path.

4) 'pgLibs': List of files to bundle into `pgadmin_commons.js`. The purpose is to separate files other than browser node modules JS in `pgadmin_commons.js` and browser node modules JS in `app.bundle.js`. It will help in debugging the code during development.

Un-vendor modules:

All modules are un-vendored except:
- requireJS
- Backgrid JS - it gives reference error when importing backgrid.css from node_modules in bundle/lib.css even if the path is correct. I logged an issue:
Opened an issue

- React JS - I didn't un-vendor React JS because the pivotal developers submitted a patch to fix issue of QtWebEngine. Once the patch is merged in React repo, we will un-vendor.

Other issues faced:

1) Backbone JS: I didn't upgraded backbone JS to latest because it affects the application code mainly `Preferences module`. 

2) jQuery: I didn't upgraded it to latest because it is gives error in loading wcIframe of wcDocker in Query tool. I submit a PR.

3) Acitree - it is not available in yarn repository. logged an request
However i am managing it on my github account by forking this repo.

Specified the repo github link to `acitree` in package.json with tag to pull from 
4.5.0-rc.7. The latest version of actiree has issues so code is used form 4.5.0-rc.7 tag. 

Thanks to bestander for helping this out. link to thread

Plugins used:

​- optimize-css-assets-webpack-plugin: its job is to optimise the CSS bundle like minifying CSS and removing comments from it. [used only in production]

-  uglifyJS - It minimise the bundled JS​, and remove console statements from code. [used only in production]

- definePlugin - It helps in minimising the `React' production bundle. As react has conditional code based on 'NODE_ENV' variable. [used only in production]

- providePlugin - It makes the variable defined through out the application context. For example: jQuery object can be accessed wherever  it is used but not included in `define` calls.

- CommonsChunkPlugin - it helps in separating vendor like code, common dependent modules used in multiple modules. it extracts out in a file.

- extractTextPlugin - it is used in combination with 'css-loader' and 'style-loader'. It helps in extracting CSS and moved into files.

- webpack-bundle-analyzer - it helps in analysing the bundled JS files like size of bundled JS and which JS files are included in bundle. It is commented out. [Use only when need to analyse]

Loaders used:

​- shim-loader: It manages the module dependency, uses the same configuration used in ​requireJS

- imports-loader: It also used to loaded dependent modules which are defined in its `use` setting.

- file-loader - It helps in extracting font, image files into a separate directory.

- css-loader - Imports css files included in modules using `require` and `import` calls.

TODO:

​1) Automatically handle static and template JS files: This is already being discussed. Once it is sorted out, we will change webpack configuration accordingly.

2) Implementing Caching: I will look into this once an initial patch is commited. and later on add as improvement.

3) Source maps: It will help in debugging bundled JS code in production environment.

4) Feature tests are failing: I am currently looking into it. Query tool functionality is working fine, the issue is it is unable to find code mirror.

Steps to run:

​After applying patch: git apply --binary /path/to/patch

run `yarn install`
then run:

​In development mode:
yarn run bundle:dev

In production mode:
yarn run bundle:prod​​

Performance comparison:

On Mac's Chrome - Before bundle it was taking ~9sec to load page. After bundle it took 3.5secs (with no cache)

Please review the patch.

Thanks,
Surinder


On Wed, Jul 5, 2017 at 8:22 PM, Sarah McAlear <smcalear@pivotal.io> wrote:
Hello,


​Things to discuss:

How to differentiate between a static and template JS
​​
.

What is the advantage of webpacking templated JS? It seems as though this creates a system in which the bundled dependencies have to refer back to the backend to load the templates. 

If there is a performance win in packing templated JS then looking at it makes sense.  Otherwise it may make sense to put off until it is clear that the templated files should be dealt with by either de-templating them or bundling them where there is a clear reason.

However, we're wondering about possible performance penalties with templating larger files (as opposed to templating on-demand.) Since jinja templates can execute arbitrary python, this could get time expensive and further slow things like initial page-load.
Another concern is: what happens when a template gets out of date (e.g. if browser.js had previously filled in the content for 'panel_item.content' and had been cached, would it render a new version with the new values when needed? Or is it possible that we would get old content?) 
 
Taks remaining:

​1. ​
Fix local variables which are declared without using var, have to check in each file
​ by​
 running eslint (For now, i will fix only errors which are giving error in browser).

​2. ​
Move non-template files from ’templates’ to ’static’ directory. List of
​ pending​
 modules is here: 
  • Tools (mostly all modules - 9 modules) 
  • Browser nodes - 3 modules(resource group, roles, tablespace)
  • ​About
    ​​
Also can we move 
​'​
dashboard, statistic
​s​
, preferences and help
​'​
 modules inside misc to preserve modularity as pgAdmin is modular
​ ?​

Is there anything from a organization stance you discussed in the previous email that needs to be done to make this usable and consistent?


Thanks,

George & Sarah 




--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company



Re: [pgAdmin4]: Webpacking of static JS/CSS

From
Robert Eckhardt
Date:
The last email on this chain from Surinder says that it isn't working on IE and he will submit another patch. Are we missing that patch? Would you like us to look at the previous patch?

-- Rob


On Jul 11, 2017 11:37 AM, "Dave Page" <dave.page@enterprisedb.com> wrote:
Pivotal team; you guys are far more familiar with webpack etc. than we are; could you review please?

Thanks!

On Tue, Jul 11, 2017 at 4:24 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

1) Created a separated patch for Un-vendored libraries and another one related to webpack bundle files so it is easy to review.

2) Removed commented out code and dead code.

3) Feature test cases are fixed. All are running.
I have to add `time.sleep` of 1 second in method 'fill_codemirror_area_with' as sometimes it work and sometimes don't.

4. Removed unused libraries from package.json

Please find updated patch and review.

Thanks,
Surinder



On Tue, Jul 11, 2017 at 3:11 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

This patch doesn't work in windows IE 11 due to error `'Promise' is undefined
​`​.

The dependency package 'babel-polyfill' is required to run ES6 code with webpack and has to load before at entry point of app.
related thread

Please find updated patch.

Thanks
Surinder

On Tue, Jul 11, 2017 at 2:13 PM, Dave Page <dave.page@enterprisedb.com> wrote:
Nice!

On Tue, Jul 11, 2017 at 9:42 AM, Neel Patel <neel.patel@enterprisedb.com> wrote:
Hi Dave,

I have tested Surinder's patch in my local machine with Qt 5.9.1 Webkit on Windows and we are getting improvements with this patch. :)

Below performance tested with Qt 5.9.1 with annulen webkit v 5.212.0 Alpha2.

Before Webpack patch:-

It took ~20 seconds. This 20 seconds includes when user double click on application ( timing of python server start + page load )

After Webpack patch:-

It took ~13 seconds ( timing of python server start + page load ).

We got ~7 seconds improvement with webpack on same machine & same Qt configuration and this will be useful in windows performance issue as well :)

Thanks,
Neel  Patel

On Tue, Jul 11, 2017 at 1:27 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi 

I found this patch won't work on IE, so i have fixed it and will send updated patch.

On Tue, Jul 11, 2017 at 1:25 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:

On Tue, Jul 11, 2017 at 1:22 PM, Dave Page <dpage@pgadmin.org> wrote:
When you say "un-vendorising", do you mean removing them from the vendor directory and adding them to packages.json so they're pulled in via npm/yarn? (if so, good :-) )
​Yes.​

On Tue, Jul 11, 2017 at 7:34 AM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

Detailed changes:

​1) Created a file bundle/app.js which loads `browser.js`  and then 'tools.datagrid' and its other dependents are configured to load in '`imports-loader`

2) bundle/codemirror.js - it generates a bundled JS which is used wherever required in various modules.

3) file_utils.js - It bundles the file manager's utility JS file and loaded from `file manager/index.html`

4) lib_css - It contains list of vendor CSS to be bundled.

5) pgadmin_css - It contains list of overrides CSS which are bundled and which has to load after vendors CSS is loaded.

Various Webpack configuration parameters:

1) externals: List of template files to be loaded dynamically when a call is made by dependent modules. These files are excluded from the bundled JS.

2) resolve > alias - This resolves the path of module JS which is referred in other modules; For example: 
'backbone': path.join(__dirname, './node_modules/backbone/backbone')

3) `backbone` is used in 'define(['backbone'])' calls and its path is resolved to above absolute path.

4) 'pgLibs': List of files to bundle into `pgadmin_commons.js`. The purpose is to separate files other than browser node modules JS in `pgadmin_commons.js` and browser node modules JS in `app.bundle.js`. It will help in debugging the code during development.

Un-vendor modules:

All modules are un-vendored except:
- requireJS
- Backgrid JS - it gives reference error when importing backgrid.css from node_modules in bundle/lib.css even if the path is correct. I logged an issue:
Opened an issue

- React JS - I didn't un-vendor React JS because the pivotal developers submitted a patch to fix issue of QtWebEngine. Once the patch is merged in React repo, we will un-vendor.

Other issues faced:

1) Backbone JS: I didn't upgraded backbone JS to latest because it affects the application code mainly `Preferences module`. 

2) jQuery: I didn't upgraded it to latest because it is gives error in loading wcIframe of wcDocker in Query tool. I submit a PR.

3) Acitree - it is not available in yarn repository. logged an request
However i am managing it on my github account by forking this repo.

Specified the repo github link to `acitree` in package.json with tag to pull from 
4.5.0-rc.7. The latest version of actiree has issues so code is used form 4.5.0-rc.7 tag. 

Thanks to bestander for helping this out. link to thread

Plugins used:

​- optimize-css-assets-webpack-plugin: its job is to optimise the CSS bundle like minifying CSS and removing comments from it. [used only in production]

-  uglifyJS - It minimise the bundled JS​, and remove console statements from code. [used only in production]

- definePlugin - It helps in minimising the `React' production bundle. As react has conditional code based on 'NODE_ENV' variable. [used only in production]

- providePlugin - It makes the variable defined through out the application context. For example: jQuery object can be accessed wherever  it is used but not included in `define` calls.

- CommonsChunkPlugin - it helps in separating vendor like code, common dependent modules used in multiple modules. it extracts out in a file.

- extractTextPlugin - it is used in combination with 'css-loader' and 'style-loader'. It helps in extracting CSS and moved into files.

- webpack-bundle-analyzer - it helps in analysing the bundled JS files like size of bundled JS and which JS files are included in bundle. It is commented out. [Use only when need to analyse]

Loaders used:

​- shim-loader: It manages the module dependency, uses the same configuration used in ​requireJS

- imports-loader: It also used to loaded dependent modules which are defined in its `use` setting.

- file-loader - It helps in extracting font, image files into a separate directory.

- css-loader - Imports css files included in modules using `require` and `import` calls.

TODO:

​1) Automatically handle static and template JS files: This is already being discussed. Once it is sorted out, we will change webpack configuration accordingly.

2) Implementing Caching: I will look into this once an initial patch is commited. and later on add as improvement.

3) Source maps: It will help in debugging bundled JS code in production environment.

4) Feature tests are failing: I am currently looking into it. Query tool functionality is working fine, the issue is it is unable to find code mirror.

Steps to run:

​After applying patch: git apply --binary /path/to/patch

run `yarn install`
then run:

​In development mode:
yarn run bundle:dev

In production mode:
yarn run bundle:prod​​

Performance comparison:

On Mac's Chrome - Before bundle it was taking ~9sec to load page. After bundle it took 3.5secs (with no cache)

Please review the patch.

Thanks,
Surinder


On Wed, Jul 5, 2017 at 8:22 PM, Sarah McAlear <smcalear@pivotal.io> wrote:
Hello,


​Things to discuss:

How to differentiate between a static and template JS
​​
.

What is the advantage of webpacking templated JS? It seems as though this creates a system in which the bundled dependencies have to refer back to the backend to load the templates. 

If there is a performance win in packing templated JS then looking at it makes sense.  Otherwise it may make sense to put off until it is clear that the templated files should be dealt with by either de-templating them or bundling them where there is a clear reason.

However, we're wondering about possible performance penalties with templating larger files (as opposed to templating on-demand.) Since jinja templates can execute arbitrary python, this could get time expensive and further slow things like initial page-load.
Another concern is: what happens when a template gets out of date (e.g. if browser.js had previously filled in the content for 'panel_item.content' and had been cached, would it render a new version with the new values when needed? Or is it possible that we would get old content?) 
 
Taks remaining:

​1. ​
Fix local variables which are declared without using var, have to check in each file
​ by​
 running eslint (For now, i will fix only errors which are giving error in browser).

​2. ​
Move non-template files from ’templates’ to ’static’ directory. List of
​ pending​
 modules is here: 
  • Tools (mostly all modules - 9 modules) 
  • Browser nodes - 3 modules(resource group, roles, tablespace)
  • ​About
    ​​
Also can we move 
​'​
dashboard, statistic
​s​
, preferences and help
​'​
 modules inside misc to preserve modularity as pgAdmin is modular
​ ?​

Is there anything from a organization stance you discussed in the previous email that needs to be done to make this usable and consistent?


Thanks,

George & Sarah 




--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company






--
Dave Page
VP, Chief Architect, Tools & Installers
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake





--
Dave Page
VP, Chief Architect, Tools & Installers
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

Re: [pgAdmin4]: Webpacking of static JS/CSS

From
Surinder Kumar
Date:
On Tue, Jul 11, 2017 at 10:18 PM, Robert Eckhardt <reckhardt@pivotal.io> wrote:
The last email on this chain from Surinder says that it isn't working on IE and he will submit another patch. Are we missing that patch? Would you like us to look at the previous patch?
​Yes previous patch includes fix related to IE.​

-- Rob


On Jul 11, 2017 11:37 AM, "Dave Page" <dave.page@enterprisedb.com> wrote:
Pivotal team; you guys are far more familiar with webpack etc. than we are; could you review please?

Thanks!

On Tue, Jul 11, 2017 at 4:24 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

1) Created a separated patch for Un-vendored libraries and another one related to webpack bundle files so it is easy to review.

2) Removed commented out code and dead code.

3) Feature test cases are fixed. All are running.
I have to add `time.sleep` of 1 second in method 'fill_codemirror_area_with' as sometimes it work and sometimes don't.

4. Removed unused libraries from package.json

Please find updated patch and review.

Thanks,
Surinder



On Tue, Jul 11, 2017 at 3:11 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

This patch doesn't work in windows IE 11 due to error `'Promise' is undefined
​`​.

The dependency package 'babel-polyfill' is required to run ES6 code with webpack and has to load before at entry point of app.
related thread

Please find updated patch.

Thanks
Surinder

On Tue, Jul 11, 2017 at 2:13 PM, Dave Page <dave.page@enterprisedb.com> wrote:
Nice!

On Tue, Jul 11, 2017 at 9:42 AM, Neel Patel <neel.patel@enterprisedb.com> wrote:
Hi Dave,

I have tested Surinder's patch in my local machine with Qt 5.9.1 Webkit on Windows and we are getting improvements with this patch. :)

Below performance tested with Qt 5.9.1 with annulen webkit v 5.212.0 Alpha2.

Before Webpack patch:-

It took ~20 seconds. This 20 seconds includes when user double click on application ( timing of python server start + page load )

After Webpack patch:-

It took ~13 seconds ( timing of python server start + page load ).

We got ~7 seconds improvement with webpack on same machine & same Qt configuration and this will be useful in windows performance issue as well :)

Thanks,
Neel  Patel

On Tue, Jul 11, 2017 at 1:27 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi 

I found this patch won't work on IE, so i have fixed it and will send updated patch.

On Tue, Jul 11, 2017 at 1:25 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:

On Tue, Jul 11, 2017 at 1:22 PM, Dave Page <dpage@pgadmin.org> wrote:
When you say "un-vendorising", do you mean removing them from the vendor directory and adding them to packages.json so they're pulled in via npm/yarn? (if so, good :-) )
​Yes.​

On Tue, Jul 11, 2017 at 7:34 AM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

Detailed changes:

​1) Created a file bundle/app.js which loads `browser.js`  and then 'tools.datagrid' and its other dependents are configured to load in '`imports-loader`

2) bundle/codemirror.js - it generates a bundled JS which is used wherever required in various modules.

3) file_utils.js - It bundles the file manager's utility JS file and loaded from `file manager/index.html`

4) lib_css - It contains list of vendor CSS to be bundled.

5) pgadmin_css - It contains list of overrides CSS which are bundled and which has to load after vendors CSS is loaded.

Various Webpack configuration parameters:

1) externals: List of template files to be loaded dynamically when a call is made by dependent modules. These files are excluded from the bundled JS.

2) resolve > alias - This resolves the path of module JS which is referred in other modules; For example: 
'backbone': path.join(__dirname, './node_modules/backbone/backbone')

3) `backbone` is used in 'define(['backbone'])' calls and its path is resolved to above absolute path.

4) 'pgLibs': List of files to bundle into `pgadmin_commons.js`. The purpose is to separate files other than browser node modules JS in `pgadmin_commons.js` and browser node modules JS in `app.bundle.js`. It will help in debugging the code during development.

Un-vendor modules:

All modules are un-vendored except:
- requireJS
- Backgrid JS - it gives reference error when importing backgrid.css from node_modules in bundle/lib.css even if the path is correct. I logged an issue:
Opened an issue

- React JS - I didn't un-vendor React JS because the pivotal developers submitted a patch to fix issue of QtWebEngine. Once the patch is merged in React repo, we will un-vendor.

Other issues faced:

1) Backbone JS: I didn't upgraded backbone JS to latest because it affects the application code mainly `Preferences module`. 

2) jQuery: I didn't upgraded it to latest because it is gives error in loading wcIframe of wcDocker in Query tool. I submit a PR.

3) Acitree - it is not available in yarn repository. logged an request
However i am managing it on my github account by forking this repo.

Specified the repo github link to `acitree` in package.json with tag to pull from 
4.5.0-rc.7. The latest version of actiree has issues so code is used form 4.5.0-rc.7 tag. 

Thanks to bestander for helping this out. link to thread

Plugins used:

​- optimize-css-assets-webpack-plugin: its job is to optimise the CSS bundle like minifying CSS and removing comments from it. [used only in production]

-  uglifyJS - It minimise the bundled JS​, and remove console statements from code. [used only in production]

- definePlugin - It helps in minimising the `React' production bundle. As react has conditional code based on 'NODE_ENV' variable. [used only in production]

- providePlugin - It makes the variable defined through out the application context. For example: jQuery object can be accessed wherever  it is used but not included in `define` calls.

- CommonsChunkPlugin - it helps in separating vendor like code, common dependent modules used in multiple modules. it extracts out in a file.

- extractTextPlugin - it is used in combination with 'css-loader' and 'style-loader'. It helps in extracting CSS and moved into files.

- webpack-bundle-analyzer - it helps in analysing the bundled JS files like size of bundled JS and which JS files are included in bundle. It is commented out. [Use only when need to analyse]

Loaders used:

​- shim-loader: It manages the module dependency, uses the same configuration used in ​requireJS

- imports-loader: It also used to loaded dependent modules which are defined in its `use` setting.

- file-loader - It helps in extracting font, image files into a separate directory.

- css-loader - Imports css files included in modules using `require` and `import` calls.

TODO:

​1) Automatically handle static and template JS files: This is already being discussed. Once it is sorted out, we will change webpack configuration accordingly.

2) Implementing Caching: I will look into this once an initial patch is commited. and later on add as improvement.

3) Source maps: It will help in debugging bundled JS code in production environment.

4) Feature tests are failing: I am currently looking into it. Query tool functionality is working fine, the issue is it is unable to find code mirror.

Steps to run:

​After applying patch: git apply --binary /path/to/patch

run `yarn install`
then run:

​In development mode:
yarn run bundle:dev

In production mode:
yarn run bundle:prod​​

Performance comparison:

On Mac's Chrome - Before bundle it was taking ~9sec to load page. After bundle it took 3.5secs (with no cache)

Please review the patch.

Thanks,
Surinder


On Wed, Jul 5, 2017 at 8:22 PM, Sarah McAlear <smcalear@pivotal.io> wrote:
Hello,


​Things to discuss:

How to differentiate between a static and template JS
​​
.

What is the advantage of webpacking templated JS? It seems as though this creates a system in which the bundled dependencies have to refer back to the backend to load the templates. 

If there is a performance win in packing templated JS then looking at it makes sense.  Otherwise it may make sense to put off until it is clear that the templated files should be dealt with by either de-templating them or bundling them where there is a clear reason.

However, we're wondering about possible performance penalties with templating larger files (as opposed to templating on-demand.) Since jinja templates can execute arbitrary python, this could get time expensive and further slow things like initial page-load.
Another concern is: what happens when a template gets out of date (e.g. if browser.js had previously filled in the content for 'panel_item.content' and had been cached, would it render a new version with the new values when needed? Or is it possible that we would get old content?) 
 
Taks remaining:

​1. ​
Fix local variables which are declared without using var, have to check in each file
​ by​
 running eslint (For now, i will fix only errors which are giving error in browser).

​2. ​
Move non-template files from ’templates’ to ’static’ directory. List of
​ pending​
 modules is here: 
  • Tools (mostly all modules - 9 modules) 
  • Browser nodes - 3 modules(resource group, roles, tablespace)
  • ​About
    ​​
Also can we move 
​'​
dashboard, statistic
​s​
, preferences and help
​'​
 modules inside misc to preserve modularity as pgAdmin is modular
​ ?​

Is there anything from a organization stance you discussed in the previous email that needs to be done to make this usable and consistent?


Thanks,

George & Sarah 




--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company






--
Dave Page
VP, Chief Architect, Tools & Installers
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake





--
Dave Page
VP, Chief Architect, Tools & Installers
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

Re: [pgAdmin4]: Webpacking of static JS/CSS

From
Robert Eckhardt
Date:
Surinder, 

Sorry I'm missing the email can you tell me the name please. 

-- Rob

On Tue, Jul 11, 2017 at 12:51 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
On Tue, Jul 11, 2017 at 10:18 PM, Robert Eckhardt <reckhardt@pivotal.io> wrote:
The last email on this chain from Surinder says that it isn't working on IE and he will submit another patch. Are we missing that patch? Would you like us to look at the previous patch?
​Yes previous patch includes fix related to IE.​

-- Rob


On Jul 11, 2017 11:37 AM, "Dave Page" <dave.page@enterprisedb.com> wrote:
Pivotal team; you guys are far more familiar with webpack etc. than we are; could you review please?

Thanks!

On Tue, Jul 11, 2017 at 4:24 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

1) Created a separated patch for Un-vendored libraries and another one related to webpack bundle files so it is easy to review.

2) Removed commented out code and dead code.

3) Feature test cases are fixed. All are running.
I have to add `time.sleep` of 1 second in method 'fill_codemirror_area_with' as sometimes it work and sometimes don't.

4. Removed unused libraries from package.json

Please find updated patch and review.

Thanks,
Surinder



On Tue, Jul 11, 2017 at 3:11 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

This patch doesn't work in windows IE 11 due to error `'Promise' is undefined
​`​.

The dependency package 'babel-polyfill' is required to run ES6 code with webpack and has to load before at entry point of app.
related thread

Please find updated patch.

Thanks
Surinder

On Tue, Jul 11, 2017 at 2:13 PM, Dave Page <dave.page@enterprisedb.com> wrote:
Nice!

On Tue, Jul 11, 2017 at 9:42 AM, Neel Patel <neel.patel@enterprisedb.com> wrote:
Hi Dave,

I have tested Surinder's patch in my local machine with Qt 5.9.1 Webkit on Windows and we are getting improvements with this patch. :)

Below performance tested with Qt 5.9.1 with annulen webkit v 5.212.0 Alpha2.

Before Webpack patch:-

It took ~20 seconds. This 20 seconds includes when user double click on application ( timing of python server start + page load )

After Webpack patch:-

It took ~13 seconds ( timing of python server start + page load ).

We got ~7 seconds improvement with webpack on same machine & same Qt configuration and this will be useful in windows performance issue as well :)

Thanks,
Neel  Patel

On Tue, Jul 11, 2017 at 1:27 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi 

I found this patch won't work on IE, so i have fixed it and will send updated patch.

On Tue, Jul 11, 2017 at 1:25 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:

On Tue, Jul 11, 2017 at 1:22 PM, Dave Page <dpage@pgadmin.org> wrote:
When you say "un-vendorising", do you mean removing them from the vendor directory and adding them to packages.json so they're pulled in via npm/yarn? (if so, good :-) )
​Yes.​

On Tue, Jul 11, 2017 at 7:34 AM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

Detailed changes:

​1) Created a file bundle/app.js which loads `browser.js`  and then 'tools.datagrid' and its other dependents are configured to load in '`imports-loader`

2) bundle/codemirror.js - it generates a bundled JS which is used wherever required in various modules.

3) file_utils.js - It bundles the file manager's utility JS file and loaded from `file manager/index.html`

4) lib_css - It contains list of vendor CSS to be bundled.

5) pgadmin_css - It contains list of overrides CSS which are bundled and which has to load after vendors CSS is loaded.

Various Webpack configuration parameters:

1) externals: List of template files to be loaded dynamically when a call is made by dependent modules. These files are excluded from the bundled JS.

2) resolve > alias - This resolves the path of module JS which is referred in other modules; For example: 
'backbone': path.join(__dirname, './node_modules/backbone/backbone')

3) `backbone` is used in 'define(['backbone'])' calls and its path is resolved to above absolute path.

4) 'pgLibs': List of files to bundle into `pgadmin_commons.js`. The purpose is to separate files other than browser node modules JS in `pgadmin_commons.js` and browser node modules JS in `app.bundle.js`. It will help in debugging the code during development.

Un-vendor modules:

All modules are un-vendored except:
- requireJS
- Backgrid JS - it gives reference error when importing backgrid.css from node_modules in bundle/lib.css even if the path is correct. I logged an issue:
Opened an issue

- React JS - I didn't un-vendor React JS because the pivotal developers submitted a patch to fix issue of QtWebEngine. Once the patch is merged in React repo, we will un-vendor.

Other issues faced:

1) Backbone JS: I didn't upgraded backbone JS to latest because it affects the application code mainly `Preferences module`. 

2) jQuery: I didn't upgraded it to latest because it is gives error in loading wcIframe of wcDocker in Query tool. I submit a PR.

3) Acitree - it is not available in yarn repository. logged an request
However i am managing it on my github account by forking this repo.

Specified the repo github link to `acitree` in package.json with tag to pull from 
4.5.0-rc.7. The latest version of actiree has issues so code is used form 4.5.0-rc.7 tag. 

Thanks to bestander for helping this out. link to thread

Plugins used:

​- optimize-css-assets-webpack-plugin: its job is to optimise the CSS bundle like minifying CSS and removing comments from it. [used only in production]

-  uglifyJS - It minimise the bundled JS​, and remove console statements from code. [used only in production]

- definePlugin - It helps in minimising the `React' production bundle. As react has conditional code based on 'NODE_ENV' variable. [used only in production]

- providePlugin - It makes the variable defined through out the application context. For example: jQuery object can be accessed wherever  it is used but not included in `define` calls.

- CommonsChunkPlugin - it helps in separating vendor like code, common dependent modules used in multiple modules. it extracts out in a file.

- extractTextPlugin - it is used in combination with 'css-loader' and 'style-loader'. It helps in extracting CSS and moved into files.

- webpack-bundle-analyzer - it helps in analysing the bundled JS files like size of bundled JS and which JS files are included in bundle. It is commented out. [Use only when need to analyse]

Loaders used:

​- shim-loader: It manages the module dependency, uses the same configuration used in ​requireJS

- imports-loader: It also used to loaded dependent modules which are defined in its `use` setting.

- file-loader - It helps in extracting font, image files into a separate directory.

- css-loader - Imports css files included in modules using `require` and `import` calls.

TODO:

​1) Automatically handle static and template JS files: This is already being discussed. Once it is sorted out, we will change webpack configuration accordingly.

2) Implementing Caching: I will look into this once an initial patch is commited. and later on add as improvement.

3) Source maps: It will help in debugging bundled JS code in production environment.

4) Feature tests are failing: I am currently looking into it. Query tool functionality is working fine, the issue is it is unable to find code mirror.

Steps to run:

​After applying patch: git apply --binary /path/to/patch

run `yarn install`
then run:

​In development mode:
yarn run bundle:dev

In production mode:
yarn run bundle:prod​​

Performance comparison:

On Mac's Chrome - Before bundle it was taking ~9sec to load page. After bundle it took 3.5secs (with no cache)

Please review the patch.

Thanks,
Surinder


On Wed, Jul 5, 2017 at 8:22 PM, Sarah McAlear <smcalear@pivotal.io> wrote:
Hello,


​Things to discuss:

How to differentiate between a static and template JS
​​
.

What is the advantage of webpacking templated JS? It seems as though this creates a system in which the bundled dependencies have to refer back to the backend to load the templates. 

If there is a performance win in packing templated JS then looking at it makes sense.  Otherwise it may make sense to put off until it is clear that the templated files should be dealt with by either de-templating them or bundling them where there is a clear reason.

However, we're wondering about possible performance penalties with templating larger files (as opposed to templating on-demand.) Since jinja templates can execute arbitrary python, this could get time expensive and further slow things like initial page-load.
Another concern is: what happens when a template gets out of date (e.g. if browser.js had previously filled in the content for 'panel_item.content' and had been cached, would it render a new version with the new values when needed? Or is it possible that we would get old content?) 
 
Taks remaining:

​1. ​
Fix local variables which are declared without using var, have to check in each file
​ by​
 running eslint (For now, i will fix only errors which are giving error in browser).

​2. ​
Move non-template files from ’templates’ to ’static’ directory. List of
​ pending​
 modules is here: 
  • Tools (mostly all modules - 9 modules) 
  • Browser nodes - 3 modules(resource group, roles, tablespace)
  • ​About
    ​​
Also can we move 
​'​
dashboard, statistic
​s​
, preferences and help
​'​
 modules inside misc to preserve modularity as pgAdmin is modular
​ ?​

Is there anything from a organization stance you discussed in the previous email that needs to be done to make this usable and consistent?


Thanks,

George & Sarah 




--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company






--
Dave Page
VP, Chief Architect, Tools & Installers
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake





--
Dave Page
VP, Chief Architect, Tools & Installers
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake


Re: [pgAdmin4]: Webpacking of static JS/CSS

From
Surinder Kumar
Date:
Robert, 

I have attached two patches:

1. webpack_bundles.patch

2. unvendor_files.patch

in this email chain.

If you didn't received those patches possibly due to large size of patch, let me know i will send again.


On Tue, Jul 11, 2017 at 10:24 PM, Robert Eckhardt <reckhardt@pivotal.io> wrote:
Surinder, 

Sorry I'm missing the email can you tell me the name please. 

-- Rob

On Tue, Jul 11, 2017 at 12:51 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
On Tue, Jul 11, 2017 at 10:18 PM, Robert Eckhardt <reckhardt@pivotal.io> wrote:
The last email on this chain from Surinder says that it isn't working on IE and he will submit another patch. Are we missing that patch? Would you like us to look at the previous patch?
​Yes previous patch includes fix related to IE.​

-- Rob


On Jul 11, 2017 11:37 AM, "Dave Page" <dave.page@enterprisedb.com> wrote:
Pivotal team; you guys are far more familiar with webpack etc. than we are; could you review please?

Thanks!

On Tue, Jul 11, 2017 at 4:24 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

1) Created a separated patch for Un-vendored libraries and another one related to webpack bundle files so it is easy to review.

2) Removed commented out code and dead code.

3) Feature test cases are fixed. All are running.
I have to add `time.sleep` of 1 second in method 'fill_codemirror_area_with' as sometimes it work and sometimes don't.

4. Removed unused libraries from package.json

Please find updated patch and review.

Thanks,
Surinder



On Tue, Jul 11, 2017 at 3:11 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

This patch doesn't work in windows IE 11 due to error `'Promise' is undefined
​`​.

The dependency package 'babel-polyfill' is required to run ES6 code with webpack and has to load before at entry point of app.
related thread

Please find updated patch.

Thanks
Surinder

On Tue, Jul 11, 2017 at 2:13 PM, Dave Page <dave.page@enterprisedb.com> wrote:
Nice!

On Tue, Jul 11, 2017 at 9:42 AM, Neel Patel <neel.patel@enterprisedb.com> wrote:
Hi Dave,

I have tested Surinder's patch in my local machine with Qt 5.9.1 Webkit on Windows and we are getting improvements with this patch. :)

Below performance tested with Qt 5.9.1 with annulen webkit v 5.212.0 Alpha2.

Before Webpack patch:-

It took ~20 seconds. This 20 seconds includes when user double click on application ( timing of python server start + page load )

After Webpack patch:-

It took ~13 seconds ( timing of python server start + page load ).

We got ~7 seconds improvement with webpack on same machine & same Qt configuration and this will be useful in windows performance issue as well :)

Thanks,
Neel  Patel

On Tue, Jul 11, 2017 at 1:27 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi 

I found this patch won't work on IE, so i have fixed it and will send updated patch.

On Tue, Jul 11, 2017 at 1:25 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:

On Tue, Jul 11, 2017 at 1:22 PM, Dave Page <dpage@pgadmin.org> wrote:
When you say "un-vendorising", do you mean removing them from the vendor directory and adding them to packages.json so they're pulled in via npm/yarn? (if so, good :-) )
​Yes.​

On Tue, Jul 11, 2017 at 7:34 AM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

Detailed changes:

​1) Created a file bundle/app.js which loads `browser.js`  and then 'tools.datagrid' and its other dependents are configured to load in '`imports-loader`

2) bundle/codemirror.js - it generates a bundled JS which is used wherever required in various modules.

3) file_utils.js - It bundles the file manager's utility JS file and loaded from `file manager/index.html`

4) lib_css - It contains list of vendor CSS to be bundled.

5) pgadmin_css - It contains list of overrides CSS which are bundled and which has to load after vendors CSS is loaded.

Various Webpack configuration parameters:

1) externals: List of template files to be loaded dynamically when a call is made by dependent modules. These files are excluded from the bundled JS.

2) resolve > alias - This resolves the path of module JS which is referred in other modules; For example: 
'backbone': path.join(__dirname, './node_modules/backbone/backbone')

3) `backbone` is used in 'define(['backbone'])' calls and its path is resolved to above absolute path.

4) 'pgLibs': List of files to bundle into `pgadmin_commons.js`. The purpose is to separate files other than browser node modules JS in `pgadmin_commons.js` and browser node modules JS in `app.bundle.js`. It will help in debugging the code during development.

Un-vendor modules:

All modules are un-vendored except:
- requireJS
- Backgrid JS - it gives reference error when importing backgrid.css from node_modules in bundle/lib.css even if the path is correct. I logged an issue:
Opened an issue

- React JS - I didn't un-vendor React JS because the pivotal developers submitted a patch to fix issue of QtWebEngine. Once the patch is merged in React repo, we will un-vendor.

Other issues faced:

1) Backbone JS: I didn't upgraded backbone JS to latest because it affects the application code mainly `Preferences module`. 

2) jQuery: I didn't upgraded it to latest because it is gives error in loading wcIframe of wcDocker in Query tool. I submit a PR.

3) Acitree - it is not available in yarn repository. logged an request
However i am managing it on my github account by forking this repo.

Specified the repo github link to `acitree` in package.json with tag to pull from 
4.5.0-rc.7. The latest version of actiree has issues so code is used form 4.5.0-rc.7 tag. 

Thanks to bestander for helping this out. link to thread

Plugins used:

​- optimize-css-assets-webpack-plugin: its job is to optimise the CSS bundle like minifying CSS and removing comments from it. [used only in production]

-  uglifyJS - It minimise the bundled JS​, and remove console statements from code. [used only in production]

- definePlugin - It helps in minimising the `React' production bundle. As react has conditional code based on 'NODE_ENV' variable. [used only in production]

- providePlugin - It makes the variable defined through out the application context. For example: jQuery object can be accessed wherever  it is used but not included in `define` calls.

- CommonsChunkPlugin - it helps in separating vendor like code, common dependent modules used in multiple modules. it extracts out in a file.

- extractTextPlugin - it is used in combination with 'css-loader' and 'style-loader'. It helps in extracting CSS and moved into files.

- webpack-bundle-analyzer - it helps in analysing the bundled JS files like size of bundled JS and which JS files are included in bundle. It is commented out. [Use only when need to analyse]

Loaders used:

​- shim-loader: It manages the module dependency, uses the same configuration used in ​requireJS

- imports-loader: It also used to loaded dependent modules which are defined in its `use` setting.

- file-loader - It helps in extracting font, image files into a separate directory.

- css-loader - Imports css files included in modules using `require` and `import` calls.

TODO:

​1) Automatically handle static and template JS files: This is already being discussed. Once it is sorted out, we will change webpack configuration accordingly.

2) Implementing Caching: I will look into this once an initial patch is commited. and later on add as improvement.

3) Source maps: It will help in debugging bundled JS code in production environment.

4) Feature tests are failing: I am currently looking into it. Query tool functionality is working fine, the issue is it is unable to find code mirror.

Steps to run:

​After applying patch: git apply --binary /path/to/patch

run `yarn install`
then run:

​In development mode:
yarn run bundle:dev

In production mode:
yarn run bundle:prod​​

Performance comparison:

On Mac's Chrome - Before bundle it was taking ~9sec to load page. After bundle it took 3.5secs (with no cache)

Please review the patch.

Thanks,
Surinder


On Wed, Jul 5, 2017 at 8:22 PM, Sarah McAlear <smcalear@pivotal.io> wrote:
Hello,


​Things to discuss:

How to differentiate between a static and template JS
​​
.

What is the advantage of webpacking templated JS? It seems as though this creates a system in which the bundled dependencies have to refer back to the backend to load the templates. 

If there is a performance win in packing templated JS then looking at it makes sense.  Otherwise it may make sense to put off until it is clear that the templated files should be dealt with by either de-templating them or bundling them where there is a clear reason.

However, we're wondering about possible performance penalties with templating larger files (as opposed to templating on-demand.) Since jinja templates can execute arbitrary python, this could get time expensive and further slow things like initial page-load.
Another concern is: what happens when a template gets out of date (e.g. if browser.js had previously filled in the content for 'panel_item.content' and had been cached, would it render a new version with the new values when needed? Or is it possible that we would get old content?) 
 
Taks remaining:

​1. ​
Fix local variables which are declared without using var, have to check in each file
​ by​
 running eslint (For now, i will fix only errors which are giving error in browser).

​2. ​
Move non-template files from ’templates’ to ’static’ directory. List of
​ pending​
 modules is here: 
  • Tools (mostly all modules - 9 modules) 
  • Browser nodes - 3 modules(resource group, roles, tablespace)
  • ​About
    ​​
Also can we move 
​'​
dashboard, statistic
​s​
, preferences and help
​'​
 modules inside misc to preserve modularity as pgAdmin is modular
​ ?​

Is there anything from a organization stance you discussed in the previous email that needs to be done to make this usable and consistent?


Thanks,

George & Sarah 




--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company






--
Dave Page
VP, Chief Architect, Tools & Installers
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake





--
Dave Page
VP, Chief Architect, Tools & Installers
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake



Re: [pgAdmin4]: Webpacking of static JS/CSS

From
Matthew Kleiman
Date:
Hi Surinder!

We are seeing some good improvements to the codebase with this patch. It's good to see many global variable declarations were changed to local var. This patch puts the code in a good place using webpack and stop vendorizing libraries. 

Due to the large bundles being generated, the application takes a much longer time to start. This could become painful for developers. 
We noticed that what used to be the react_components.js bundle is now inside sqleditor bundle. This means that the entire react codebase is being bundled into sqleditor. It might be better to have a separate bundle for react because there may be future components built using react that are outside of sqleditor.

We've attached some changes that we did on top of the patches that you sent. This includes:
  1. We removed the package_lock.json file. It seems this is a new feature of npm version 5 that generates this lock file. However, since we are using yarn, it isn't needed. 
  2. We removed commented lines that we came across during the review.
  3. We removed console logs that we came across during the review.
  4. We changed this to the new self variable in one more place where you introduced self in handle_query_output_keyboard_event.js.
  5. We included a notation for /* eslint-env node */ instead of /* eslint-disable */ in webpack.shim.js. This allows linting of the file while ignoring node-specific things.
Good Work!
Matt and João 


On Tue, Jul 11, 2017 at 12:59 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Robert, 

I have attached two patches:

1. webpack_bundles.patch

2. unvendor_files.patch

in this email chain.

If you didn't received those patches possibly due to large size of patch, let me know i will send again.


On Tue, Jul 11, 2017 at 10:24 PM, Robert Eckhardt <reckhardt@pivotal.io> wrote:
Surinder, 

Sorry I'm missing the email can you tell me the name please. 

-- Rob

On Tue, Jul 11, 2017 at 12:51 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
On Tue, Jul 11, 2017 at 10:18 PM, Robert Eckhardt <reckhardt@pivotal.io> wrote:
The last email on this chain from Surinder says that it isn't working on IE and he will submit another patch. Are we missing that patch? Would you like us to look at the previous patch?
​Yes previous patch includes fix related to IE.​

-- Rob


On Jul 11, 2017 11:37 AM, "Dave Page" <dave.page@enterprisedb.com> wrote:
Pivotal team; you guys are far more familiar with webpack etc. than we are; could you review please?

Thanks!

On Tue, Jul 11, 2017 at 4:24 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

1) Created a separated patch for Un-vendored libraries and another one related to webpack bundle files so it is easy to review.

2) Removed commented out code and dead code.

3) Feature test cases are fixed. All are running.
I have to add `time.sleep` of 1 second in method 'fill_codemirror_area_with' as sometimes it work and sometimes don't.

4. Removed unused libraries from package.json

Please find updated patch and review.

Thanks,
Surinder



On Tue, Jul 11, 2017 at 3:11 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

This patch doesn't work in windows IE 11 due to error `'Promise' is undefined
​`​.

The dependency package 'babel-polyfill' is required to run ES6 code with webpack and has to load before at entry point of app.
related thread

Please find updated patch.

Thanks
Surinder

On Tue, Jul 11, 2017 at 2:13 PM, Dave Page <dave.page@enterprisedb.com> wrote:
Nice!

On Tue, Jul 11, 2017 at 9:42 AM, Neel Patel <neel.patel@enterprisedb.com> wrote:
Hi Dave,

I have tested Surinder's patch in my local machine with Qt 5.9.1 Webkit on Windows and we are getting improvements with this patch. :)

Below performance tested with Qt 5.9.1 with annulen webkit v 5.212.0 Alpha2.

Before Webpack patch:-

It took ~20 seconds. This 20 seconds includes when user double click on application ( timing of python server start + page load )

After Webpack patch:-

It took ~13 seconds ( timing of python server start + page load ).

We got ~7 seconds improvement with webpack on same machine & same Qt configuration and this will be useful in windows performance issue as well :)

Thanks,
Neel  Patel

On Tue, Jul 11, 2017 at 1:27 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi 

I found this patch won't work on IE, so i have fixed it and will send updated patch.

On Tue, Jul 11, 2017 at 1:25 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:

On Tue, Jul 11, 2017 at 1:22 PM, Dave Page <dpage@pgadmin.org> wrote:
When you say "un-vendorising", do you mean removing them from the vendor directory and adding them to packages.json so they're pulled in via npm/yarn? (if so, good :-) )
​Yes.​

On Tue, Jul 11, 2017 at 7:34 AM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

Detailed changes:

​1) Created a file bundle/app.js which loads `browser.js`  and then 'tools.datagrid' and its other dependents are configured to load in '`imports-loader`

2) bundle/codemirror.js - it generates a bundled JS which is used wherever required in various modules.

3) file_utils.js - It bundles the file manager's utility JS file and loaded from `file manager/index.html`

4) lib_css - It contains list of vendor CSS to be bundled.

5) pgadmin_css - It contains list of overrides CSS which are bundled and which has to load after vendors CSS is loaded.

Various Webpack configuration parameters:

1) externals: List of template files to be loaded dynamically when a call is made by dependent modules. These files are excluded from the bundled JS.

2) resolve > alias - This resolves the path of module JS which is referred in other modules; For example: 
'backbone': path.join(__dirname, './node_modules/backbone/backbone')

3) `backbone` is used in 'define(['backbone'])' calls and its path is resolved to above absolute path.

4) 'pgLibs': List of files to bundle into `pgadmin_commons.js`. The purpose is to separate files other than browser node modules JS in `pgadmin_commons.js` and browser node modules JS in `app.bundle.js`. It will help in debugging the code during development.

Un-vendor modules:

All modules are un-vendored except:
- requireJS
- Backgrid JS - it gives reference error when importing backgrid.css from node_modules in bundle/lib.css even if the path is correct. I logged an issue:
Opened an issue

- React JS - I didn't un-vendor React JS because the pivotal developers submitted a patch to fix issue of QtWebEngine. Once the patch is merged in React repo, we will un-vendor.

Other issues faced:

1) Backbone JS: I didn't upgraded backbone JS to latest because it affects the application code mainly `Preferences module`. 

2) jQuery: I didn't upgraded it to latest because it is gives error in loading wcIframe of wcDocker in Query tool. I submit a PR.

3) Acitree - it is not available in yarn repository. logged an request
However i am managing it on my github account by forking this repo.

Specified the repo github link to `acitree` in package.json with tag to pull from 
4.5.0-rc.7. The latest version of actiree has issues so code is used form 4.5.0-rc.7 tag. 

Thanks to bestander for helping this out. link to thread

Plugins used:

​- optimize-css-assets-webpack-plugin: its job is to optimise the CSS bundle like minifying CSS and removing comments from it. [used only in production]

-  uglifyJS - It minimise the bundled JS​, and remove console statements from code. [used only in production]

- definePlugin - It helps in minimising the `React' production bundle. As react has conditional code based on 'NODE_ENV' variable. [used only in production]

- providePlugin - It makes the variable defined through out the application context. For example: jQuery object can be accessed wherever  it is used but not included in `define` calls.

- CommonsChunkPlugin - it helps in separating vendor like code, common dependent modules used in multiple modules. it extracts out in a file.

- extractTextPlugin - it is used in combination with 'css-loader' and 'style-loader'. It helps in extracting CSS and moved into files.

- webpack-bundle-analyzer - it helps in analysing the bundled JS files like size of bundled JS and which JS files are included in bundle. It is commented out. [Use only when need to analyse]

Loaders used:

​- shim-loader: It manages the module dependency, uses the same configuration used in ​requireJS

- imports-loader: It also used to loaded dependent modules which are defined in its `use` setting.

- file-loader - It helps in extracting font, image files into a separate directory.

- css-loader - Imports css files included in modules using `require` and `import` calls.

TODO:

​1) Automatically handle static and template JS files: This is already being discussed. Once it is sorted out, we will change webpack configuration accordingly.

2) Implementing Caching: I will look into this once an initial patch is commited. and later on add as improvement.

3) Source maps: It will help in debugging bundled JS code in production environment.

4) Feature tests are failing: I am currently looking into it. Query tool functionality is working fine, the issue is it is unable to find code mirror.

Steps to run:

​After applying patch: git apply --binary /path/to/patch

run `yarn install`
then run:

​In development mode:
yarn run bundle:dev

In production mode:
yarn run bundle:prod​​

Performance comparison:

On Mac's Chrome - Before bundle it was taking ~9sec to load page. After bundle it took 3.5secs (with no cache)

Please review the patch.

Thanks,
Surinder


On Wed, Jul 5, 2017 at 8:22 PM, Sarah McAlear <smcalear@pivotal.io> wrote:
Hello,


​Things to discuss:

How to differentiate between a static and template JS
​​
.

What is the advantage of webpacking templated JS? It seems as though this creates a system in which the bundled dependencies have to refer back to the backend to load the templates. 

If there is a performance win in packing templated JS then looking at it makes sense.  Otherwise it may make sense to put off until it is clear that the templated files should be dealt with by either de-templating them or bundling them where there is a clear reason.

However, we're wondering about possible performance penalties with templating larger files (as opposed to templating on-demand.) Since jinja templates can execute arbitrary python, this could get time expensive and further slow things like initial page-load.
Another concern is: what happens when a template gets out of date (e.g. if browser.js had previously filled in the content for 'panel_item.content' and had been cached, would it render a new version with the new values when needed? Or is it possible that we would get old content?) 
 
Taks remaining:

​1. ​
Fix local variables which are declared without using var, have to check in each file
​ by​
 running eslint (For now, i will fix only errors which are giving error in browser).

​2. ​
Move non-template files from ’templates’ to ’static’ directory. List of
​ pending​
 modules is here: 
  • Tools (mostly all modules - 9 modules) 
  • Browser nodes - 3 modules(resource group, roles, tablespace)
  • ​About
    ​​
Also can we move 
​'​
dashboard, statistic
​s​
, preferences and help
​'​
 modules inside misc to preserve modularity as pgAdmin is modular
​ ?​

Is there anything from a organization stance you discussed in the previous email that needs to be done to make this usable and consistent?


Thanks,

George & Sarah 




--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company






--
Dave Page
VP, Chief Architect, Tools & Installers
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake





--
Dave Page
VP, Chief Architect, Tools & Installers
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake




Attachment

Re: [pgAdmin4]: Webpacking of static JS/CSS

From
Ashesh Vashi
Date:
On Tue, Jul 11, 2017 at 10:29 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Robert, 

I have attached two patches:

1. webpack_bundles.patch

2. unvendor_files.patch

in this email chain.

If you didn't received those patches possibly due to large size of patch, let me know i will send again.
Dave,

I don't see these two patches too in the mail-chain.
By any chance, it is stalled.

-- Thanks, Ashesh 


On Tue, Jul 11, 2017 at 10:24 PM, Robert Eckhardt <reckhardt@pivotal.io> wrote:
Surinder, 

Sorry I'm missing the email can you tell me the name please. 

-- Rob

On Tue, Jul 11, 2017 at 12:51 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
On Tue, Jul 11, 2017 at 10:18 PM, Robert Eckhardt <reckhardt@pivotal.io> wrote:
The last email on this chain from Surinder says that it isn't working on IE and he will submit another patch. Are we missing that patch? Would you like us to look at the previous patch?
​Yes previous patch includes fix related to IE.​

-- Rob


On Jul 11, 2017 11:37 AM, "Dave Page" <dave.page@enterprisedb.com> wrote:
Pivotal team; you guys are far more familiar with webpack etc. than we are; could you review please?

Thanks!

On Tue, Jul 11, 2017 at 4:24 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

1) Created a separated patch for Un-vendored libraries and another one related to webpack bundle files so it is easy to review.

2) Removed commented out code and dead code.

3) Feature test cases are fixed. All are running.
I have to add `time.sleep` of 1 second in method 'fill_codemirror_area_with' as sometimes it work and sometimes don't.

4. Removed unused libraries from package.json

Please find updated patch and review.

Thanks,
Surinder



On Tue, Jul 11, 2017 at 3:11 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

This patch doesn't work in windows IE 11 due to error `'Promise' is undefined
​`​.

The dependency package 'babel-polyfill' is required to run ES6 code with webpack and has to load before at entry point of app.
related thread

Please find updated patch.

Thanks
Surinder

On Tue, Jul 11, 2017 at 2:13 PM, Dave Page <dave.page@enterprisedb.com> wrote:
Nice!

On Tue, Jul 11, 2017 at 9:42 AM, Neel Patel <neel.patel@enterprisedb.com> wrote:
Hi Dave,

I have tested Surinder's patch in my local machine with Qt 5.9.1 Webkit on Windows and we are getting improvements with this patch. :)

Below performance tested with Qt 5.9.1 with annulen webkit v 5.212.0 Alpha2.

Before Webpack patch:-

It took ~20 seconds. This 20 seconds includes when user double click on application ( timing of python server start + page load )

After Webpack patch:-

It took ~13 seconds ( timing of python server start + page load ).

We got ~7 seconds improvement with webpack on same machine & same Qt configuration and this will be useful in windows performance issue as well :)

Thanks,
Neel  Patel

On Tue, Jul 11, 2017 at 1:27 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi 

I found this patch won't work on IE, so i have fixed it and will send updated patch.

On Tue, Jul 11, 2017 at 1:25 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:

On Tue, Jul 11, 2017 at 1:22 PM, Dave Page <dpage@pgadmin.org> wrote:
When you say "un-vendorising", do you mean removing them from the vendor directory and adding them to packages.json so they're pulled in via npm/yarn? (if so, good :-) )
​Yes.​

On Tue, Jul 11, 2017 at 7:34 AM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

Detailed changes:

​1) Created a file bundle/app.js which loads `browser.js`  and then 'tools.datagrid' and its other dependents are configured to load in '`imports-loader`

2) bundle/codemirror.js - it generates a bundled JS which is used wherever required in various modules.

3) file_utils.js - It bundles the file manager's utility JS file and loaded from `file manager/index.html`

4) lib_css - It contains list of vendor CSS to be bundled.

5) pgadmin_css - It contains list of overrides CSS which are bundled and which has to load after vendors CSS is loaded.

Various Webpack configuration parameters:

1) externals: List of template files to be loaded dynamically when a call is made by dependent modules. These files are excluded from the bundled JS.

2) resolve > alias - This resolves the path of module JS which is referred in other modules; For example: 
'backbone': path.join(__dirname, './node_modules/backbone/backbone')

3) `backbone` is used in 'define(['backbone'])' calls and its path is resolved to above absolute path.

4) 'pgLibs': List of files to bundle into `pgadmin_commons.js`. The purpose is to separate files other than browser node modules JS in `pgadmin_commons.js` and browser node modules JS in `app.bundle.js`. It will help in debugging the code during development.

Un-vendor modules:

All modules are un-vendored except:
- requireJS
- Backgrid JS - it gives reference error when importing backgrid.css from node_modules in bundle/lib.css even if the path is correct. I logged an issue:
Opened an issue

- React JS - I didn't un-vendor React JS because the pivotal developers submitted a patch to fix issue of QtWebEngine. Once the patch is merged in React repo, we will un-vendor.

Other issues faced:

1) Backbone JS: I didn't upgraded backbone JS to latest because it affects the application code mainly `Preferences module`. 

2) jQuery: I didn't upgraded it to latest because it is gives error in loading wcIframe of wcDocker in Query tool. I submit a PR.

3) Acitree - it is not available in yarn repository. logged an request
However i am managing it on my github account by forking this repo.

Specified the repo github link to `acitree` in package.json with tag to pull from 
4.5.0-rc.7. The latest version of actiree has issues so code is used form 4.5.0-rc.7 tag. 

Thanks to bestander for helping this out. link to thread

Plugins used:

​- optimize-css-assets-webpack-plugin: its job is to optimise the CSS bundle like minifying CSS and removing comments from it. [used only in production]

-  uglifyJS - It minimise the bundled JS​, and remove console statements from code. [used only in production]

- definePlugin - It helps in minimising the `React' production bundle. As react has conditional code based on 'NODE_ENV' variable. [used only in production]

- providePlugin - It makes the variable defined through out the application context. For example: jQuery object can be accessed wherever  it is used but not included in `define` calls.

- CommonsChunkPlugin - it helps in separating vendor like code, common dependent modules used in multiple modules. it extracts out in a file.

- extractTextPlugin - it is used in combination with 'css-loader' and 'style-loader'. It helps in extracting CSS and moved into files.

- webpack-bundle-analyzer - it helps in analysing the bundled JS files like size of bundled JS and which JS files are included in bundle. It is commented out. [Use only when need to analyse]

Loaders used:

​- shim-loader: It manages the module dependency, uses the same configuration used in ​requireJS

- imports-loader: It also used to loaded dependent modules which are defined in its `use` setting.

- file-loader - It helps in extracting font, image files into a separate directory.

- css-loader - Imports css files included in modules using `require` and `import` calls.

TODO:

​1) Automatically handle static and template JS files: This is already being discussed. Once it is sorted out, we will change webpack configuration accordingly.

2) Implementing Caching: I will look into this once an initial patch is commited. and later on add as improvement.

3) Source maps: It will help in debugging bundled JS code in production environment.

4) Feature tests are failing: I am currently looking into it. Query tool functionality is working fine, the issue is it is unable to find code mirror.

Steps to run:

​After applying patch: git apply --binary /path/to/patch

run `yarn install`
then run:

​In development mode:
yarn run bundle:dev

In production mode:
yarn run bundle:prod​​

Performance comparison:

On Mac's Chrome - Before bundle it was taking ~9sec to load page. After bundle it took 3.5secs (with no cache)

Please review the patch.

Thanks,
Surinder


On Wed, Jul 5, 2017 at 8:22 PM, Sarah McAlear <smcalear@pivotal.io> wrote:
Hello,


​Things to discuss:

How to differentiate between a static and template JS
​​
.

What is the advantage of webpacking templated JS? It seems as though this creates a system in which the bundled dependencies have to refer back to the backend to load the templates. 

If there is a performance win in packing templated JS then looking at it makes sense.  Otherwise it may make sense to put off until it is clear that the templated files should be dealt with by either de-templating them or bundling them where there is a clear reason.

However, we're wondering about possible performance penalties with templating larger files (as opposed to templating on-demand.) Since jinja templates can execute arbitrary python, this could get time expensive and further slow things like initial page-load.
Another concern is: what happens when a template gets out of date (e.g. if browser.js had previously filled in the content for 'panel_item.content' and had been cached, would it render a new version with the new values when needed? Or is it possible that we would get old content?) 
 
Taks remaining:

​1. ​
Fix local variables which are declared without using var, have to check in each file
​ by​
 running eslint (For now, i will fix only errors which are giving error in browser).

​2. ​
Move non-template files from ’templates’ to ’static’ directory. List of
​ pending​
 modules is here: 
  • Tools (mostly all modules - 9 modules) 
  • Browser nodes - 3 modules(resource group, roles, tablespace)
  • ​About
    ​​
Also can we move 
​'​
dashboard, statistic
​s​
, preferences and help
​'​
 modules inside misc to preserve modularity as pgAdmin is modular
​ ?​

Is there anything from a organization stance you discussed in the previous email that needs to be done to make this usable and consistent?


Thanks,

George & Sarah 




--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company






--
Dave Page
VP, Chief Architect, Tools & Installers
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake





--
Dave Page
VP, Chief Architect, Tools & Installers
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake




Re: [pgAdmin4]: Webpacking of static JS/CSS

From
Dave Page
Date:


On 12 Jul 2017, at 02:56, Ashesh Vashi <ashesh.vashi@enterprisedb.com> wrote:

On Tue, Jul 11, 2017 at 10:29 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Robert, 

I have attached two patches:

1. webpack_bundles.patch

2. unvendor_files.patch

in this email chain.

If you didn't received those patches possibly due to large size of patch, let me know i will send again.
Dave,

I don't see these two patches too in the mail-chain.
By any chance, it is stalled.

They're way too big for the mailing lists (>13MB). Please have Surinder send you them privately.


-- Thanks, Ashesh 


On Tue, Jul 11, 2017 at 10:24 PM, Robert Eckhardt <reckhardt@pivotal.io> wrote:
Surinder, 

Sorry I'm missing the email can you tell me the name please. 

-- Rob

On Tue, Jul 11, 2017 at 12:51 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
On Tue, Jul 11, 2017 at 10:18 PM, Robert Eckhardt <reckhardt@pivotal.io> wrote:
The last email on this chain from Surinder says that it isn't working on IE and he will submit another patch. Are we missing that patch? Would you like us to look at the previous patch?
​Yes previous patch includes fix related to IE.​

-- Rob


On Jul 11, 2017 11:37 AM, "Dave Page" <dave.page@enterprisedb.com> wrote:
Pivotal team; you guys are far more familiar with webpack etc. than we are; could you review please?

Thanks!

On Tue, Jul 11, 2017 at 4:24 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

1) Created a separated patch for Un-vendored libraries and another one related to webpack bundle files so it is easy to review.

2) Removed commented out code and dead code.

3) Feature test cases are fixed. All are running.
I have to add `time.sleep` of 1 second in method 'fill_codemirror_area_with' as sometimes it work and sometimes don't.

4. Removed unused libraries from package.json

Please find updated patch and review.

Thanks,
Surinder



On Tue, Jul 11, 2017 at 3:11 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

This patch doesn't work in windows IE 11 due to error `'Promise' is undefined
​`​.

The dependency package 'babel-polyfill' is required to run ES6 code with webpack and has to load before at entry point of app.
related thread

Please find updated patch.

Thanks
Surinder

On Tue, Jul 11, 2017 at 2:13 PM, Dave Page <dave.page@enterprisedb.com> wrote:
Nice!

On Tue, Jul 11, 2017 at 9:42 AM, Neel Patel <neel.patel@enterprisedb.com> wrote:
Hi Dave,

I have tested Surinder's patch in my local machine with Qt 5.9.1 Webkit on Windows and we are getting improvements with this patch. :)

Below performance tested with Qt 5.9.1 with annulen webkit v 5.212.0 Alpha2.

Before Webpack patch:-

It took ~20 seconds. This 20 seconds includes when user double click on application ( timing of python server start + page load )

After Webpack patch:-

It took ~13 seconds ( timing of python server start + page load ).

We got ~7 seconds improvement with webpack on same machine & same Qt configuration and this will be useful in windows performance issue as well :)

Thanks,
Neel  Patel

On Tue, Jul 11, 2017 at 1:27 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi 

I found this patch won't work on IE, so i have fixed it and will send updated patch.

On Tue, Jul 11, 2017 at 1:25 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:

On Tue, Jul 11, 2017 at 1:22 PM, Dave Page <dpage@pgadmin.org> wrote:
When you say "un-vendorising", do you mean removing them from the vendor directory and adding them to packages.json so they're pulled in via npm/yarn? (if so, good :-) )
​Yes.​

On Tue, Jul 11, 2017 at 7:34 AM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

Detailed changes:

​1) Created a file bundle/app.js which loads `browser.js`  and then 'tools.datagrid' and its other dependents are configured to load in '`imports-loader`

2) bundle/codemirror.js - it generates a bundled JS which is used wherever required in various modules.

3) file_utils.js - It bundles the file manager's utility JS file and loaded from `file manager/index.html`

4) lib_css - It contains list of vendor CSS to be bundled.

5) pgadmin_css - It contains list of overrides CSS which are bundled and which has to load after vendors CSS is loaded.

Various Webpack configuration parameters:

1) externals: List of template files to be loaded dynamically when a call is made by dependent modules. These files are excluded from the bundled JS.

2) resolve > alias - This resolves the path of module JS which is referred in other modules; For example: 
'backbone': path.join(__dirname, './node_modules/backbone/backbone')

3) `backbone` is used in 'define(['backbone'])' calls and its path is resolved to above absolute path.

4) 'pgLibs': List of files to bundle into `pgadmin_commons.js`. The purpose is to separate files other than browser node modules JS in `pgadmin_commons.js` and browser node modules JS in `app.bundle.js`. It will help in debugging the code during development.

Un-vendor modules:

All modules are un-vendored except:
- requireJS
- Backgrid JS - it gives reference error when importing backgrid.css from node_modules in bundle/lib.css even if the path is correct. I logged an issue:
Opened an issue

- React JS - I didn't un-vendor React JS because the pivotal developers submitted a patch to fix issue of QtWebEngine. Once the patch is merged in React repo, we will un-vendor.

Other issues faced:

1) Backbone JS: I didn't upgraded backbone JS to latest because it affects the application code mainly `Preferences module`. 

2) jQuery: I didn't upgraded it to latest because it is gives error in loading wcIframe of wcDocker in Query tool. I submit a PR.

3) Acitree - it is not available in yarn repository. logged an request
However i am managing it on my github account by forking this repo.

Specified the repo github link to `acitree` in package.json with tag to pull from 
4.5.0-rc.7. The latest version of actiree has issues so code is used form 4.5.0-rc.7 tag. 

Thanks to bestander for helping this out. link to thread

Plugins used:

​- optimize-css-assets-webpack-plugin: its job is to optimise the CSS bundle like minifying CSS and removing comments from it. [used only in production]

-  uglifyJS - It minimise the bundled JS​, and remove console statements from code. [used only in production]

- definePlugin - It helps in minimising the `React' production bundle. As react has conditional code based on 'NODE_ENV' variable. [used only in production]

- providePlugin - It makes the variable defined through out the application context. For example: jQuery object can be accessed wherever  it is used but not included in `define` calls.

- CommonsChunkPlugin - it helps in separating vendor like code, common dependent modules used in multiple modules. it extracts out in a file.

- extractTextPlugin - it is used in combination with 'css-loader' and 'style-loader'. It helps in extracting CSS and moved into files.

- webpack-bundle-analyzer - it helps in analysing the bundled JS files like size of bundled JS and which JS files are included in bundle. It is commented out. [Use only when need to analyse]

Loaders used:

​- shim-loader: It manages the module dependency, uses the same configuration used in ​requireJS

- imports-loader: It also used to loaded dependent modules which are defined in its `use` setting.

- file-loader - It helps in extracting font, image files into a separate directory.

- css-loader - Imports css files included in modules using `require` and `import` calls.

TODO:

​1) Automatically handle static and template JS files: This is already being discussed. Once it is sorted out, we will change webpack configuration accordingly.

2) Implementing Caching: I will look into this once an initial patch is commited. and later on add as improvement.

3) Source maps: It will help in debugging bundled JS code in production environment.

4) Feature tests are failing: I am currently looking into it. Query tool functionality is working fine, the issue is it is unable to find code mirror.

Steps to run:

​After applying patch: git apply --binary /path/to/patch

run `yarn install`
then run:

​In development mode:
yarn run bundle:dev

In production mode:
yarn run bundle:prod​​

Performance comparison:

On Mac's Chrome - Before bundle it was taking ~9sec to load page. After bundle it took 3.5secs (with no cache)

Please review the patch.

Thanks,
Surinder


On Wed, Jul 5, 2017 at 8:22 PM, Sarah McAlear <smcalear@pivotal.io> wrote:
Hello,


​Things to discuss:

How to differentiate between a static and template JS
​​
.

What is the advantage of webpacking templated JS? It seems as though this creates a system in which the bundled dependencies have to refer back to the backend to load the templates. 

If there is a performance win in packing templated JS then looking at it makes sense.  Otherwise it may make sense to put off until it is clear that the templated files should be dealt with by either de-templating them or bundling them where there is a clear reason.

However, we're wondering about possible performance penalties with templating larger files (as opposed to templating on-demand.) Since jinja templates can execute arbitrary python, this could get time expensive and further slow things like initial page-load.
Another concern is: what happens when a template gets out of date (e.g. if browser.js had previously filled in the content for 'panel_item.content' and had been cached, would it render a new version with the new values when needed? Or is it possible that we would get old content?) 
 
Taks remaining:

​1. ​
Fix local variables which are declared without using var, have to check in each file
​ by​
 running eslint (For now, i will fix only errors which are giving error in browser).

​2. ​
Move non-template files from ’templates’ to ’static’ directory. List of
​ pending​
 modules is here: 
  • Tools (mostly all modules - 9 modules) 
  • Browser nodes - 3 modules(resource group, roles, tablespace)
  • ​About
    ​​
Also can we move 
​'​
dashboard, statistic
​s​
, preferences and help
​'​
 modules inside misc to preserve modularity as pgAdmin is modular
​ ?​

Is there anything from a organization stance you discussed in the previous email that needs to be done to make this usable and consistent?


Thanks,

George & Sarah 




--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company






--
Dave Page
VP, Chief Architect, Tools & Installers
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake





--
Dave Page
VP, Chief Architect, Tools & Installers
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake




Re: [pgAdmin4]: Webpacking of static JS/CSS

From
Surinder Kumar
Date:
​Hi

As previous patches gets stalled due to large size.
Please find attached patch.​


On Wed, Jul 12, 2017 at 12:23 PM, Dave Page <dpage@pgadmin.org> wrote:


On 12 Jul 2017, at 02:56, Ashesh Vashi <ashesh.vashi@enterprisedb.com> wrote:

On Tue, Jul 11, 2017 at 10:29 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Robert, 

I have attached two patches:

1. webpack_bundles.patch

2. unvendor_files.patch

in this email chain.

If you didn't received those patches possibly due to large size of patch, let me know i will send again.
Dave,

I don't see these two patches too in the mail-chain.
By any chance, it is stalled.

They're way too big for the mailing lists (>13MB). Please have Surinder send you them privately.
​I think the patch​`webpack_bundles.patch` needs review and its size is now around 500Kb. The other one is collection of deleted vendor files which will be required when patch will be commited. So for now i will attach first patch only.


-- Thanks, Ashesh 


On Tue, Jul 11, 2017 at 10:24 PM, Robert Eckhardt <reckhardt@pivotal.io> wrote:
Surinder, 

Sorry I'm missing the email can you tell me the name please. 

-- Rob

On Tue, Jul 11, 2017 at 12:51 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
On Tue, Jul 11, 2017 at 10:18 PM, Robert Eckhardt <reckhardt@pivotal.io> wrote:
The last email on this chain from Surinder says that it isn't working on IE and he will submit another patch. Are we missing that patch? Would you like us to look at the previous patch?
​Yes previous patch includes fix related to IE.​

-- Rob


On Jul 11, 2017 11:37 AM, "Dave Page" <dave.page@enterprisedb.com> wrote:
Pivotal team; you guys are far more familiar with webpack etc. than we are; could you review please?

Thanks!

On Tue, Jul 11, 2017 at 4:24 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

1) Created a separated patch for Un-vendored libraries and another one related to webpack bundle files so it is easy to review.

2) Removed commented out code and dead code.

3) Feature test cases are fixed. All are running.
I have to add `time.sleep` of 1 second in method 'fill_codemirror_area_with' as sometimes it work and sometimes don't.

4. Removed unused libraries from package.json

Please find updated patch and review.

Thanks,
Surinder



On Tue, Jul 11, 2017 at 3:11 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

This patch doesn't work in windows IE 11 due to error `'Promise' is undefined
​`​.

The dependency package 'babel-polyfill' is required to run ES6 code with webpack and has to load before at entry point of app.
related thread

Please find updated patch.

Thanks
Surinder

On Tue, Jul 11, 2017 at 2:13 PM, Dave Page <dave.page@enterprisedb.com> wrote:
Nice!

On Tue, Jul 11, 2017 at 9:42 AM, Neel Patel <neel.patel@enterprisedb.com> wrote:
Hi Dave,

I have tested Surinder's patch in my local machine with Qt 5.9.1 Webkit on Windows and we are getting improvements with this patch. :)

Below performance tested with Qt 5.9.1 with annulen webkit v 5.212.0 Alpha2.

Before Webpack patch:-

It took ~20 seconds. This 20 seconds includes when user double click on application ( timing of python server start + page load )

After Webpack patch:-

It took ~13 seconds ( timing of python server start + page load ).

We got ~7 seconds improvement with webpack on same machine & same Qt configuration and this will be useful in windows performance issue as well :)

Thanks,
Neel  Patel

On Tue, Jul 11, 2017 at 1:27 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi 

I found this patch won't work on IE, so i have fixed it and will send updated patch.

On Tue, Jul 11, 2017 at 1:25 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:

On Tue, Jul 11, 2017 at 1:22 PM, Dave Page <dpage@pgadmin.org> wrote:
When you say "un-vendorising", do you mean removing them from the vendor directory and adding them to packages.json so they're pulled in via npm/yarn? (if so, good :-) )
​Yes.​

On Tue, Jul 11, 2017 at 7:34 AM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

Detailed changes:

​1) Created a file bundle/app.js which loads `browser.js`  and then 'tools.datagrid' and its other dependents are configured to load in '`imports-loader`

2) bundle/codemirror.js - it generates a bundled JS which is used wherever required in various modules.

3) file_utils.js - It bundles the file manager's utility JS file and loaded from `file manager/index.html`

4) lib_css - It contains list of vendor CSS to be bundled.

5) pgadmin_css - It contains list of overrides CSS which are bundled and which has to load after vendors CSS is loaded.

Various Webpack configuration parameters:

1) externals: List of template files to be loaded dynamically when a call is made by dependent modules. These files are excluded from the bundled JS.

2) resolve > alias - This resolves the path of module JS which is referred in other modules; For example: 
'backbone': path.join(__dirname, './node_modules/backbone/backbone')

3) `backbone` is used in 'define(['backbone'])' calls and its path is resolved to above absolute path.

4) 'pgLibs': List of files to bundle into `pgadmin_commons.js`. The purpose is to separate files other than browser node modules JS in `pgadmin_commons.js` and browser node modules JS in `app.bundle.js`. It will help in debugging the code during development.

Un-vendor modules:

All modules are un-vendored except:
- requireJS
- Backgrid JS - it gives reference error when importing backgrid.css from node_modules in bundle/lib.css even if the path is correct. I logged an issue:
Opened an issue

- React JS - I didn't un-vendor React JS because the pivotal developers submitted a patch to fix issue of QtWebEngine. Once the patch is merged in React repo, we will un-vendor.

Other issues faced:

1) Backbone JS: I didn't upgraded backbone JS to latest because it affects the application code mainly `Preferences module`. 

2) jQuery: I didn't upgraded it to latest because it is gives error in loading wcIframe of wcDocker in Query tool. I submit a PR.

3) Acitree - it is not available in yarn repository. logged an request
However i am managing it on my github account by forking this repo.

Specified the repo github link to `acitree` in package.json with tag to pull from 
4.5.0-rc.7. The latest version of actiree has issues so code is used form 4.5.0-rc.7 tag. 

Thanks to bestander for helping this out. link to thread

Plugins used:

​- optimize-css-assets-webpack-plugin: its job is to optimise the CSS bundle like minifying CSS and removing comments from it. [used only in production]

-  uglifyJS - It minimise the bundled JS​, and remove console statements from code. [used only in production]

- definePlugin - It helps in minimising the `React' production bundle. As react has conditional code based on 'NODE_ENV' variable. [used only in production]

- providePlugin - It makes the variable defined through out the application context. For example: jQuery object can be accessed wherever  it is used but not included in `define` calls.

- CommonsChunkPlugin - it helps in separating vendor like code, common dependent modules used in multiple modules. it extracts out in a file.

- extractTextPlugin - it is used in combination with 'css-loader' and 'style-loader'. It helps in extracting CSS and moved into files.

- webpack-bundle-analyzer - it helps in analysing the bundled JS files like size of bundled JS and which JS files are included in bundle. It is commented out. [Use only when need to analyse]

Loaders used:

​- shim-loader: It manages the module dependency, uses the same configuration used in ​requireJS

- imports-loader: It also used to loaded dependent modules which are defined in its `use` setting.

- file-loader - It helps in extracting font, image files into a separate directory.

- css-loader - Imports css files included in modules using `require` and `import` calls.

TODO:

​1) Automatically handle static and template JS files: This is already being discussed. Once it is sorted out, we will change webpack configuration accordingly.

2) Implementing Caching: I will look into this once an initial patch is commited. and later on add as improvement.

3) Source maps: It will help in debugging bundled JS code in production environment.

4) Feature tests are failing: I am currently looking into it. Query tool functionality is working fine, the issue is it is unable to find code mirror.

Steps to run:

​After applying patch: git apply --binary /path/to/patch

run `yarn install`
then run:

​In development mode:
yarn run bundle:dev

In production mode:
yarn run bundle:prod​​

Performance comparison:

On Mac's Chrome - Before bundle it was taking ~9sec to load page. After bundle it took 3.5secs (with no cache)

Please review the patch.

Thanks,
Surinder


On Wed, Jul 5, 2017 at 8:22 PM, Sarah McAlear <smcalear@pivotal.io> wrote:
Hello,


​Things to discuss:

How to differentiate between a static and template JS
​​
.

What is the advantage of webpacking templated JS? It seems as though this creates a system in which the bundled dependencies have to refer back to the backend to load the templates. 

If there is a performance win in packing templated JS then looking at it makes sense.  Otherwise it may make sense to put off until it is clear that the templated files should be dealt with by either de-templating them or bundling them where there is a clear reason.

However, we're wondering about possible performance penalties with templating larger files (as opposed to templating on-demand.) Since jinja templates can execute arbitrary python, this could get time expensive and further slow things like initial page-load.
Another concern is: what happens when a template gets out of date (e.g. if browser.js had previously filled in the content for 'panel_item.content' and had been cached, would it render a new version with the new values when needed? Or is it possible that we would get old content?) 
 
Taks remaining:

​1. ​
Fix local variables which are declared without using var, have to check in each file
​ by​
 running eslint (For now, i will fix only errors which are giving error in browser).

​2. ​
Move non-template files from ’templates’ to ’static’ directory. List of
​ pending​
 modules is here: 
  • Tools (mostly all modules - 9 modules) 
  • Browser nodes - 3 modules(resource group, roles, tablespace)
  • ​About
    ​​
Also can we move 
​'​
dashboard, statistic
​s​
, preferences and help
​'​
 modules inside misc to preserve modularity as pgAdmin is modular
​ ?​

Is there anything from a organization stance you discussed in the previous email that needs to be done to make this usable and consistent?


Thanks,

George & Sarah 




--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company






--
Dave Page
VP, Chief Architect, Tools & Installers
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake





--
Dave Page
VP, Chief Architect, Tools & Installers
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake





Attachment

Re: [pgAdmin4]: Webpacking of static JS/CSS

From
Surinder Kumar
Date:
Hi

While this patch is in review, i have not attached patch for "unvendored libs" which is around 10MB in size and needs not to review.
Attached is the patch which can be reviewed.

Changes:

1) As React will be used in other modules, so the React components codebase is bundled into "vendor.js". the sqleditor.js bundle size is now just 239kb(excluding react)

If a module needs React package as dependency, the user can write:

`import React from 'react' or

`require('react')`

in respective module.

2) As Matt earlier pointed, application takes longer to start while generating bundle files. I used "hard-source-webpack-plugin" to reduce the build time and couple of other optimisations.

- Initially webpack build time: 25.13 seconds

- After removing 'pgadmin/browser' from ' resolve > modules, the time taken to build is reduced to 24.74 seconds

​- ​
After downgrading css-loader@0.28.0 to css-loader@0.14.0
​, it took 22.44 seconds.

​The older version of css-loader doesn't have other  packages like 'post-css', so it doesn't take much time to build.

While "hard-source-webpack-plugin" manages the cache and stores it into 'generated/.cache/<env><hash>/' path.​ So when the build is run  for the first time, it takes ~22 seconds.`

When run for second time, it takes 2-3 seconds to build. it builds only the chunks which have changes.

When build in production mode, initially it took ~47 seconds.

On second run, it took ~22 seconds. the only change between development and production builds are "CommonChunks" and "UglifyJS" plugins which are used for production build. So, "hard-source-webpack-plugin" doesn't seems to support build with these two optimisation plugins.

Please review updated patch.

Thanks,
Surinder


  

On Wed, Jul 12, 2017 at 12:31 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
​Hi

As previous patches gets stalled due to large size.
Please find attached patch.​


On Wed, Jul 12, 2017 at 12:23 PM, Dave Page <dpage@pgadmin.org> wrote:


On 12 Jul 2017, at 02:56, Ashesh Vashi <ashesh.vashi@enterprisedb.com> wrote:

On Tue, Jul 11, 2017 at 10:29 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Robert, 

I have attached two patches:

1. webpack_bundles.patch

2. unvendor_files.patch

in this email chain.

If you didn't received those patches possibly due to large size of patch, let me know i will send again.
Dave,

I don't see these two patches too in the mail-chain.
By any chance, it is stalled.

They're way too big for the mailing lists (>13MB). Please have Surinder send you them privately.
​I think the patch​`webpack_bundles.patch` needs review and its size is now around 500Kb. The other one is collection of deleted vendor files which will be required when patch will be commited. So for now i will attach first patch only.


-- Thanks, Ashesh 


On Tue, Jul 11, 2017 at 10:24 PM, Robert Eckhardt <reckhardt@pivotal.io> wrote:
Surinder, 

Sorry I'm missing the email can you tell me the name please. 

-- Rob

On Tue, Jul 11, 2017 at 12:51 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
On Tue, Jul 11, 2017 at 10:18 PM, Robert Eckhardt <reckhardt@pivotal.io> wrote:
The last email on this chain from Surinder says that it isn't working on IE and he will submit another patch. Are we missing that patch? Would you like us to look at the previous patch?
​Yes previous patch includes fix related to IE.​

-- Rob


On Jul 11, 2017 11:37 AM, "Dave Page" <dave.page@enterprisedb.com> wrote:
Pivotal team; you guys are far more familiar with webpack etc. than we are; could you review please?

Thanks!

On Tue, Jul 11, 2017 at 4:24 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

1) Created a separated patch for Un-vendored libraries and another one related to webpack bundle files so it is easy to review.

2) Removed commented out code and dead code.

3) Feature test cases are fixed. All are running.
I have to add `time.sleep` of 1 second in method 'fill_codemirror_area_with' as sometimes it work and sometimes don't.

4. Removed unused libraries from package.json

Please find updated patch and review.

Thanks,
Surinder



On Tue, Jul 11, 2017 at 3:11 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

This patch doesn't work in windows IE 11 due to error `'Promise' is undefined
​`​.

The dependency package 'babel-polyfill' is required to run ES6 code with webpack and has to load before at entry point of app.
related thread

Please find updated patch.

Thanks
Surinder

On Tue, Jul 11, 2017 at 2:13 PM, Dave Page <dave.page@enterprisedb.com> wrote:
Nice!

On Tue, Jul 11, 2017 at 9:42 AM, Neel Patel <neel.patel@enterprisedb.com> wrote:
Hi Dave,

I have tested Surinder's patch in my local machine with Qt 5.9.1 Webkit on Windows and we are getting improvements with this patch. :)

Below performance tested with Qt 5.9.1 with annulen webkit v 5.212.0 Alpha2.

Before Webpack patch:-

It took ~20 seconds. This 20 seconds includes when user double click on application ( timing of python server start + page load )

After Webpack patch:-

It took ~13 seconds ( timing of python server start + page load ).

We got ~7 seconds improvement with webpack on same machine & same Qt configuration and this will be useful in windows performance issue as well :)

Thanks,
Neel  Patel

On Tue, Jul 11, 2017 at 1:27 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi 

I found this patch won't work on IE, so i have fixed it and will send updated patch.

On Tue, Jul 11, 2017 at 1:25 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:

On Tue, Jul 11, 2017 at 1:22 PM, Dave Page <dpage@pgadmin.org> wrote:
When you say "un-vendorising", do you mean removing them from the vendor directory and adding them to packages.json so they're pulled in via npm/yarn? (if so, good :-) )
​Yes.​

On Tue, Jul 11, 2017 at 7:34 AM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

Detailed changes:

​1) Created a file bundle/app.js which loads `browser.js`  and then 'tools.datagrid' and its other dependents are configured to load in '`imports-loader`

2) bundle/codemirror.js - it generates a bundled JS which is used wherever required in various modules.

3) file_utils.js - It bundles the file manager's utility JS file and loaded from `file manager/index.html`

4) lib_css - It contains list of vendor CSS to be bundled.

5) pgadmin_css - It contains list of overrides CSS which are bundled and which has to load after vendors CSS is loaded.

Various Webpack configuration parameters:

1) externals: List of template files to be loaded dynamically when a call is made by dependent modules. These files are excluded from the bundled JS.

2) resolve > alias - This resolves the path of module JS which is referred in other modules; For example: 
'backbone': path.join(__dirname, './node_modules/backbone/backbone')

3) `backbone` is used in 'define(['backbone'])' calls and its path is resolved to above absolute path.

4) 'pgLibs': List of files to bundle into `pgadmin_commons.js`. The purpose is to separate files other than browser node modules JS in `pgadmin_commons.js` and browser node modules JS in `app.bundle.js`. It will help in debugging the code during development.

Un-vendor modules:

All modules are un-vendored except:
- requireJS
- Backgrid JS - it gives reference error when importing backgrid.css from node_modules in bundle/lib.css even if the path is correct. I logged an issue:
Opened an issue

- React JS - I didn't un-vendor React JS because the pivotal developers submitted a patch to fix issue of QtWebEngine. Once the patch is merged in React repo, we will un-vendor.

Other issues faced:

1) Backbone JS: I didn't upgraded backbone JS to latest because it affects the application code mainly `Preferences module`. 

2) jQuery: I didn't upgraded it to latest because it is gives error in loading wcIframe of wcDocker in Query tool. I submit a PR.

3) Acitree - it is not available in yarn repository. logged an request
However i am managing it on my github account by forking this repo.

Specified the repo github link to `acitree` in package.json with tag to pull from 
4.5.0-rc.7. The latest version of actiree has issues so code is used form 4.5.0-rc.7 tag. 

Thanks to bestander for helping this out. link to thread

Plugins used:

​- optimize-css-assets-webpack-plugin: its job is to optimise the CSS bundle like minifying CSS and removing comments from it. [used only in production]

-  uglifyJS - It minimise the bundled JS​, and remove console statements from code. [used only in production]

- definePlugin - It helps in minimising the `React' production bundle. As react has conditional code based on 'NODE_ENV' variable. [used only in production]

- providePlugin - It makes the variable defined through out the application context. For example: jQuery object can be accessed wherever  it is used but not included in `define` calls.

- CommonsChunkPlugin - it helps in separating vendor like code, common dependent modules used in multiple modules. it extracts out in a file.

- extractTextPlugin - it is used in combination with 'css-loader' and 'style-loader'. It helps in extracting CSS and moved into files.

- webpack-bundle-analyzer - it helps in analysing the bundled JS files like size of bundled JS and which JS files are included in bundle. It is commented out. [Use only when need to analyse]

Loaders used:

​- shim-loader: It manages the module dependency, uses the same configuration used in ​requireJS

- imports-loader: It also used to loaded dependent modules which are defined in its `use` setting.

- file-loader - It helps in extracting font, image files into a separate directory.

- css-loader - Imports css files included in modules using `require` and `import` calls.

TODO:

​1) Automatically handle static and template JS files: This is already being discussed. Once it is sorted out, we will change webpack configuration accordingly.

2) Implementing Caching: I will look into this once an initial patch is commited. and later on add as improvement.

3) Source maps: It will help in debugging bundled JS code in production environment.

4) Feature tests are failing: I am currently looking into it. Query tool functionality is working fine, the issue is it is unable to find code mirror.

Steps to run:

​After applying patch: git apply --binary /path/to/patch

run `yarn install`
then run:

​In development mode:
yarn run bundle:dev

In production mode:
yarn run bundle:prod​​

Performance comparison:

On Mac's Chrome - Before bundle it was taking ~9sec to load page. After bundle it took 3.5secs (with no cache)

Please review the patch.

Thanks,
Surinder


On Wed, Jul 5, 2017 at 8:22 PM, Sarah McAlear <smcalear@pivotal.io> wrote:
Hello,


​Things to discuss:

How to differentiate between a static and template JS
​​
.

What is the advantage of webpacking templated JS? It seems as though this creates a system in which the bundled dependencies have to refer back to the backend to load the templates. 

If there is a performance win in packing templated JS then looking at it makes sense.  Otherwise it may make sense to put off until it is clear that the templated files should be dealt with by either de-templating them or bundling them where there is a clear reason.

However, we're wondering about possible performance penalties with templating larger files (as opposed to templating on-demand.) Since jinja templates can execute arbitrary python, this could get time expensive and further slow things like initial page-load.
Another concern is: what happens when a template gets out of date (e.g. if browser.js had previously filled in the content for 'panel_item.content' and had been cached, would it render a new version with the new values when needed? Or is it possible that we would get old content?) 
 
Taks remaining:

​1. ​
Fix local variables which are declared without using var, have to check in each file
​ by​
 running eslint (For now, i will fix only errors which are giving error in browser).

​2. ​
Move non-template files from ’templates’ to ’static’ directory. List of
​ pending​
 modules is here: 
  • Tools (mostly all modules - 9 modules) 
  • Browser nodes - 3 modules(resource group, roles, tablespace)
  • ​About
    ​​
Also can we move 
​'​
dashboard, statistic
​s​
, preferences and help
​'​
 modules inside misc to preserve modularity as pgAdmin is modular
​ ?​

Is there anything from a organization stance you discussed in the previous email that needs to be done to make this usable and consistent?


Thanks,

George & Sarah 




--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company






--
Dave Page
VP, Chief Architect, Tools & Installers
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake





--
Dave Page
VP, Chief Architect, Tools & Installers
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake






Attachment

Re: [pgAdmin4]: Webpacking of static JS/CSS

From
Dave Page
Date:
Hi

On Wed, Jul 12, 2017 at 4:04 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

While this patch is in review, i have not attached patch for "unvendored libs" which is around 10MB in size and needs not to review.
Attached is the patch which can be reviewed.

Changes:

1) As React will be used in other modules, so the React components codebase is bundled into "vendor.js". the sqleditor.js bundle size is now just 239kb(excluding react)

If a module needs React package as dependency, the user can write:

`import React from 'react' or

`require('react')`

in respective module.

2) As Matt earlier pointed, application takes longer to start while generating bundle files. I used "hard-source-webpack-plugin" to reduce the build time and couple of other optimisations.

- Initially webpack build time: 25.13 seconds

- After removing 'pgadmin/browser' from ' resolve > modules, the time taken to build is reduced to 24.74 seconds

​- ​
After downgrading css-loader@0.28.0 to css-loader@0.14.0
​, it took 22.44 seconds.

​The older version of css-loader doesn't have other  packages like 'post-css', so it doesn't take much time to build.

While "hard-source-webpack-plugin" manages the cache and stores it into 'generated/.cache/<env><hash>/' path.​ So when the build is run  for the first time, it takes ~22 seconds.`

When run for second time, it takes 2-3 seconds to build. it builds only the chunks which have changes.

When build in production mode, initially it took ~47 seconds.

On second run, it took ~22 seconds. the only change between development and production builds are "CommonChunks" and "UglifyJS" plugins which are used for production build. So, "hard-source-webpack-plugin" doesn't seems to support build with these two optimisation plugins.

Please review updated patch.

Works beautifully in the browser and in the runtime; the speedup is significant. Excellent work!

However, the regression tests fail miserably :-(

ERROR in ./node_modules/slickgrid/slick.core.js
Module not found: Error: Can't resolve 'jquery.ui' in '/Users/dpage/git/pgadmin4/web/node_modules/slickgrid'
 @ ./node_modules/slickgrid/slick.core.js 3:12-32 11:12-32
 @ ./pgadmin/static/bundle/slickgrid.js

ERROR in ./node_modules/slickgrid/slick.core.js
Module not found: Error: Can't resolve 'jquery.event.drag' in '/Users/dpage/git/pgadmin4/web/node_modules/slickgrid'
 @ ./node_modules/slickgrid/slick.core.js 6:20-48 14:20-48
 @ ./pgadmin/static/bundle/slickgrid.js

ERROR in ./node_modules/slickgrid/slick.core.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/node_modules/slickgrid'
 @ ./node_modules/slickgrid/slick.core.js 1:0-17
 @ ./pgadmin/static/bundle/slickgrid.js

ERROR in ./node_modules/slickgrid/slick.grid.js
Module not found: Error: Can't resolve 'jquery.ui' in '/Users/dpage/git/pgadmin4/web/node_modules/slickgrid'
 @ ./node_modules/slickgrid/slick.grid.js 3:12-32
 @ ./pgadmin/static/bundle/slickgrid.js

ERROR in ./node_modules/slickgrid/slick.grid.js
Module not found: Error: Can't resolve 'jquery.event.drag' in '/Users/dpage/git/pgadmin4/web/node_modules/slickgrid'
 @ ./node_modules/slickgrid/slick.grid.js 6:20-48
 @ ./pgadmin/static/bundle/slickgrid.js

ERROR in ./node_modules/slickgrid/slick.grid.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/node_modules/slickgrid'
 @ ./node_modules/slickgrid/slick.grid.js 1:0-17
 @ ./pgadmin/static/bundle/slickgrid.js

ERROR in ./node_modules/slickgrid/slick.dataview.js
Module not found: Error: Can't resolve 'jquery.ui' in '/Users/dpage/git/pgadmin4/web/node_modules/slickgrid'
 @ ./node_modules/slickgrid/slick.dataview.js 3:12-32
 @ ./pgadmin/static/bundle/slickgrid.js

ERROR in ./node_modules/slickgrid/slick.dataview.js
Module not found: Error: Can't resolve 'jquery.event.drag' in '/Users/dpage/git/pgadmin4/web/node_modules/slickgrid'
 @ ./node_modules/slickgrid/slick.dataview.js 6:20-48
 @ ./pgadmin/static/bundle/slickgrid.js

ERROR in ./node_modules/slickgrid/slick.dataview.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/node_modules/slickgrid'
 @ ./node_modules/slickgrid/slick.dataview.js 1:0-17
 @ ./pgadmin/static/bundle/slickgrid.js

ERROR in ./node_modules/slickgrid/slick.editors.js
Module not found: Error: Can't resolve 'jquery.ui' in '/Users/dpage/git/pgadmin4/web/node_modules/slickgrid'
 @ ./node_modules/slickgrid/slick.editors.js 3:12-32
 @ ./pgadmin/static/bundle/slickgrid.js

ERROR in ./node_modules/slickgrid/slick.editors.js
Module not found: Error: Can't resolve 'jquery.event.drag' in '/Users/dpage/git/pgadmin4/web/node_modules/slickgrid'
 @ ./node_modules/slickgrid/slick.editors.js 6:20-48
 @ ./pgadmin/static/bundle/slickgrid.js

ERROR in ./node_modules/slickgrid/slick.editors.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/node_modules/slickgrid'
 @ ./node_modules/slickgrid/slick.editors.js 1:0-17
 @ ./pgadmin/static/bundle/slickgrid.js

ERROR in ./node_modules/slickgrid/slick.formatters.js
Module not found: Error: Can't resolve 'jquery.ui' in '/Users/dpage/git/pgadmin4/web/node_modules/slickgrid'
 @ ./node_modules/slickgrid/slick.formatters.js 3:12-32
 @ ./pgadmin/static/bundle/slickgrid.js

ERROR in ./node_modules/slickgrid/slick.formatters.js
Module not found: Error: Can't resolve 'jquery.event.drag' in '/Users/dpage/git/pgadmin4/web/node_modules/slickgrid'
 @ ./node_modules/slickgrid/slick.formatters.js 6:20-48
 @ ./pgadmin/static/bundle/slickgrid.js

ERROR in ./node_modules/slickgrid/slick.formatters.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/node_modules/slickgrid'
 @ ./node_modules/slickgrid/slick.formatters.js 1:0-17
 @ ./pgadmin/static/bundle/slickgrid.js

ERROR in ./node_modules/slickgrid/lib/jquery.event.drag-2.3.0.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/node_modules/slickgrid/lib'
 @ ./node_modules/slickgrid/lib/jquery.event.drag-2.3.0.js 1:0-17
 @ ./pgadmin/static/bundle/slickgrid.js

ERROR in ./node_modules/slickgrid/plugins/slick.autotooltips.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/node_modules/slickgrid/plugins'
 @ ./node_modules/slickgrid/plugins/slick.autotooltips.js 1:0-17
 @ ./pgadmin/static/bundle/slickgrid.js

ERROR in ./node_modules/slickgrid/plugins/slick.cellrangedecorator.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/node_modules/slickgrid/plugins'
 @ ./node_modules/slickgrid/plugins/slick.cellrangedecorator.js 1:0-17
 @ ./pgadmin/static/bundle/slickgrid.js

ERROR in ./node_modules/slickgrid/plugins/slick.cellrangeselector.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/node_modules/slickgrid/plugins'
 @ ./node_modules/slickgrid/plugins/slick.cellrangeselector.js 1:0-17
 @ ./pgadmin/static/bundle/slickgrid.js

ERROR in ./regression/javascript/code_mirror_spec.jsx
Module not found: Error: Can't resolve '../../pgadmin/static/vendor/jquery/jquery-1.11.2' in '/Users/dpage/git/pgadmin4/web/regression/javascript'
 @ ./regression/javascript/code_mirror_spec.jsx 7:14-73

ERROR in ./regression/javascript/selection/grid_selector_spec.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/regression/javascript/selection'
 @ ./regression/javascript/selection/grid_selector_spec.js 3:14-31

ERROR in ./regression/javascript/selection/copy_data_spec.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/regression/javascript/selection'
 @ ./regression/javascript/selection/copy_data_spec.js 3:14-31

ERROR in ./regression/javascript/selection/column_selector_spec.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/regression/javascript/selection'
 @ ./regression/javascript/selection/column_selector_spec.js 3:14-31

ERROR in ./regression/javascript/selection/range_selection_helper_spec.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/regression/javascript/selection'
 @ ./regression/javascript/selection/range_selection_helper_spec.js 3:14-31

ERROR in ./regression/javascript/selection/row_selector_spec.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/regression/javascript/selection'
 @ ./regression/javascript/selection/row_selector_spec.js 3:14-31

ERROR in ./regression/javascript/selection/set_staged_rows_spec.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/regression/javascript/selection'
 @ ./regression/javascript/selection/set_staged_rows_spec.js 3:14-31

ERROR in ./regression/javascript/selection/xcell_selection_model_spec.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/regression/javascript/selection'
 @ ./regression/javascript/selection/xcell_selection_model_spec.js 15:14-31

ERROR in ./regression/javascript/slickgrid/cell_selector_spec.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/regression/javascript/slickgrid'
 @ ./regression/javascript/slickgrid/cell_selector_spec.js 3:14-31

ERROR in ./regression/javascript/slickgrid/event_handlers/handle_query_output_keyboard_event_spec.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/regression/javascript/slickgrid/event_handlers'
 @ ./regression/javascript/slickgrid/event_handlers/handle_query_output_keyboard_event_spec.js 25:14-31

ERROR in ./regression/javascript/alerts/alertify_wrapper_spec.js
Module not found: Error: Can't resolve '../../../pgadmin/static/vendor/alertifyjs/alertify' in '/Users/dpage/git/pgadmin4/web/regression/javascript/alerts'
 @ ./regression/javascript/alerts/alertify_wrapper_spec.js 3:16-77

ERROR in ./pgadmin/static/js/check_node_visibility.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/pgadmin/static/js'
 @ ./pgadmin/static/js/check_node_visibility.js 12:0-42:2
 @ ./regression/javascript/check_node_visibility_spec.js

ERROR in ./pgadmin/static/js/sqleditor_utils.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/pgadmin/static/js'
 @ ./pgadmin/static/js/sqleditor_utils.js 13:0-59:2
 @ ./regression/javascript/sqleditor_utils_spec.js

ERROR in ./pgadmin/static/js/alerts/alertify_wrapper.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/pgadmin/static/js/alerts'
 @ ./pgadmin/static/js/alerts/alertify_wrapper.js 3:0-39:2
 @ ./regression/javascript/alerts/alertify_wrapper_spec.js

ERROR in ./pgadmin/static/js/alerts/alertify_wrapper.js
Module not found: Error: Can't resolve 'alertify' in '/Users/dpage/git/pgadmin4/web/pgadmin/static/js/alerts'
 @ ./pgadmin/static/js/alerts/alertify_wrapper.js 3:0-39:2
 @ ./regression/javascript/alerts/alertify_wrapper_spec.js

ERROR in ./pgadmin/static/js/selection/active_cell_capture.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/pgadmin/static/js/selection'
 @ ./pgadmin/static/js/selection/active_cell_capture.js 3:0-185:2
 @ ./regression/javascript/selection/active_cell_capture_spec.js

ERROR in ./pgadmin/static/js/selection/grid_selector.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/pgadmin/static/js/selection'
 @ ./pgadmin/static/js/selection/grid_selector.js 3:0-76:2
 @ ./regression/javascript/selection/column_selector_spec.js

ERROR in ./pgadmin/static/js/selection/xcell_selection_model.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/pgadmin/static/js/selection'
 @ ./pgadmin/static/js/selection/xcell_selection_model.js 3:0-228:2
 @ ./regression/javascript/selection/grid_selector_spec.js

ERROR in ./pgadmin/static/js/selection/clipboard.js
Module not found: Error: Can't resolve 'alertify' in '/Users/dpage/git/pgadmin4/web/pgadmin/static/js/selection'
 @ ./pgadmin/static/js/selection/clipboard.js 3:0-77:2
 @ ./regression/javascript/selection/copy_data_spec.js

ERROR in ./pgadmin/static/js/selection/copy_data.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/pgadmin/static/js/selection'
 @ ./pgadmin/static/js/selection/copy_data.js 3:0-34:2
 @ ./regression/javascript/selection/copy_data_spec.js

ERROR in ./pgadmin/static/js/selection/column_selector.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/pgadmin/static/js/selection'
 @ ./pgadmin/static/js/selection/column_selector.js 3:0-117:2
 @ ./regression/javascript/selection/column_selector_spec.js

ERROR in ./pgadmin/static/js/selection/row_selector.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/pgadmin/static/js/selection'
 @ ./pgadmin/static/js/selection/row_selector.js 3:0-92:2
 @ ./regression/javascript/selection/row_selector_spec.js

ERROR in ./pgadmin/static/js/selection/set_staged_rows.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/pgadmin/static/js/selection'
 @ ./pgadmin/static/js/selection/set_staged_rows.js 12:0-99:2
 @ ./regression/javascript/selection/set_staged_rows_spec.js

ERROR in ./pgadmin/static/jsx/history/detail/code_mirror.jsx
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/pgadmin/static/jsx/history/detail'
 @ ./pgadmin/static/jsx/history/detail/code_mirror.jsx 13:14-31
 @ ./regression/javascript/code_mirror_spec.jsx

ERROR in ./pgadmin/browser/static/js/menu.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/pgadmin/browser/static/js'
 @ ./pgadmin/browser/static/js/menu.js 5:0-424:2
 @ ./regression/javascript/browser/menu_spec.js

ERROR in ./pgadmin/static/js/check_node_visibility.js
Module not found: Error: Can't resolve 'underscore.string' in '/Users/dpage/git/pgadmin4/web/pgadmin/static/js'
 @ ./pgadmin/static/js/check_node_visibility.js 12:0-42:2
 @ ./regression/javascript/check_node_visibility_spec.js

ERROR in ./pgadmin/browser/static/js/menu.js
Module not found: Error: Can't resolve 'underscore.string' in '/Users/dpage/git/pgadmin4/web/pgadmin/browser/static/js'
 @ ./pgadmin/browser/static/js/menu.js 5:0-424:2
 @ ./regression/javascript/browser/menu_spec.js

ERROR in ./pgadmin/static/bundle/codemirror.js
Module not found: Error: Can't resolve 'pgadmin.sqlfoldcode' in '/Users/dpage/git/pgadmin4/web/pgadmin/static/bundle'
 @ ./pgadmin/static/bundle/codemirror.js 39:0-30
 @ ./pgadmin/static/jsx/history/detail/code_mirror.jsx
 @ ./regression/javascript/code_mirror_spec.jsx
webpack: Failed to compile.
PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "../../pgadmin/static/vendor/jquery/jquery-1.11.2"
  at regression/javascript/code_mirror_spec.jsx:83268

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "../../pgadmin/static/vendor/jquery/jquery-1.11.2"
  at regression/javascript/code_mirror_spec.jsx:83268

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/history/query_history_spec.jsx:50644

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/history/query_history_spec.jsx:50644

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "../../../pgadmin/static/vendor/alertifyjs/alertify"
  at regression/javascript/alerts/alertify_wrapper_spec.js:74

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "../../../pgadmin/static/vendor/alertifyjs/alertify"
  at regression/javascript/alerts/alertify_wrapper_spec.js:74

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "underscore.string"
  at regression/javascript/browser/menu_spec.js:1852

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "underscore.string"
  at regression/javascript/browser/menu_spec.js:1852

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/check_node_visibility_spec.js:1678

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/check_node_visibility_spec.js:1678

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/selection/active_cell_capture_spec.js:2318

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/selection/active_cell_capture_spec.js:2318

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/selection/column_selector_spec.js:7027

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/selection/column_selector_spec.js:7027

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/selection/copy_data_spec.js:6711

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/selection/copy_data_spec.js:6711

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/selection/grid_selector_spec.js:6834

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/selection/grid_selector_spec.js:6834

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/selection/range_boundary_navigator_spec.js:584

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/selection/range_boundary_navigator_spec.js:584

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/selection/range_selection_helper_spec.js:6198

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/selection/range_selection_helper_spec.js:6198

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/selection/row_selector_spec.js:6727

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/selection/row_selector_spec.js:6727

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/selection/set_staged_rows_spec.js:6198

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/selection/set_staged_rows_spec.js:6198

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/selection/xcell_selection_model_spec.js:7089

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/selection/xcell_selection_model_spec.js:7089

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/slickgrid/cell_selector_spec.js:6434

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/slickgrid/cell_selector_spec.js:6434

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/slickgrid/event_handlers/handle_query_output_keyboard_event_spec.js:6411

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/slickgrid/event_handlers/handle_query_output_keyboard_event_spec.js:6411

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/sqleditor_utils_spec.js:125

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/sqleditor_utils_spec.js:125

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at pgadmin/static/bundle/slickgrid.js:1008

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at pgadmin/static/bundle/slickgrid.js:1008

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Re: [pgAdmin4]: Webpacking of static JS/CSS

From
Surinder Kumar
Date:
On Mon, Jul 17, 2017 at 2:15 PM, Dave Page <dpage@pgadmin.org> wrote:
Hi

On Wed, Jul 12, 2017 at 4:04 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

While this patch is in review, i have not attached patch for "unvendored libs" which is around 10MB in size and needs not to review.
Attached is the patch which can be reviewed.

Changes:

1) As React will be used in other modules, so the React components codebase is bundled into "vendor.js". the sqleditor.js bundle size is now just 239kb(excluding react)

If a module needs React package as dependency, the user can write:

`import React from 'react' or

`require('react')`

in respective module.

2) As Matt earlier pointed, application takes longer to start while generating bundle files. I used "hard-source-webpack-plugin" to reduce the build time and couple of other optimisations.

- Initially webpack build time: 25.13 seconds

- After removing 'pgadmin/browser' from ' resolve > modules, the time taken to build is reduced to 24.74 seconds

​- ​
After downgrading css-loader@0.28.0 to css-loader@0.14.0
​, it took 22.44 seconds.

​The older version of css-loader doesn't have other  packages like 'post-css', so it doesn't take much time to build.

While "hard-source-webpack-plugin" manages the cache and stores it into 'generated/.cache/<env><hash>/' path.​ So when the build is run  for the first time, it takes ~22 seconds.`

When run for second time, it takes 2-3 seconds to build. it builds only the chunks which have changes.

When build in production mode, initially it took ~47 seconds.

On second run, it took ~22 seconds. the only change between development and production builds are "CommonChunks" and "UglifyJS" plugins which are used for production build. So, "hard-source-webpack-plugin" doesn't seems to support build with these two optimisation plugins.

Please review updated patch.

Works beautifully in the browser and in the runtime; the speedup is significant. Excellent work!

However, the regression tests fail miserably :-(
​I will fix and send updated patch.​

ERROR in ./node_modules/slickgrid/slick.core.js
Module not found: Error: Can't resolve 'jquery.ui' in '/Users/dpage/git/pgadmin4/web/node_modules/slickgrid'
 @ ./node_modules/slickgrid/slick.core.js 3:12-32 11:12-32
 @ ./pgadmin/static/bundle/slickgrid.js

ERROR in ./node_modules/slickgrid/slick.core.js
Module not found: Error: Can't resolve 'jquery.event.drag' in '/Users/dpage/git/pgadmin4/web/node_modules/slickgrid'
 @ ./node_modules/slickgrid/slick.core.js 6:20-48 14:20-48
 @ ./pgadmin/static/bundle/slickgrid.js

ERROR in ./node_modules/slickgrid/slick.core.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/node_modules/slickgrid'
 @ ./node_modules/slickgrid/slick.core.js 1:0-17
 @ ./pgadmin/static/bundle/slickgrid.js

ERROR in ./node_modules/slickgrid/slick.grid.js
Module not found: Error: Can't resolve 'jquery.ui' in '/Users/dpage/git/pgadmin4/web/node_modules/slickgrid'
 @ ./node_modules/slickgrid/slick.grid.js 3:12-32
 @ ./pgadmin/static/bundle/slickgrid.js

ERROR in ./node_modules/slickgrid/slick.grid.js
Module not found: Error: Can't resolve 'jquery.event.drag' in '/Users/dpage/git/pgadmin4/web/node_modules/slickgrid'
 @ ./node_modules/slickgrid/slick.grid.js 6:20-48
 @ ./pgadmin/static/bundle/slickgrid.js

ERROR in ./node_modules/slickgrid/slick.grid.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/node_modules/slickgrid'
 @ ./node_modules/slickgrid/slick.grid.js 1:0-17
 @ ./pgadmin/static/bundle/slickgrid.js

ERROR in ./node_modules/slickgrid/slick.dataview.js
Module not found: Error: Can't resolve 'jquery.ui' in '/Users/dpage/git/pgadmin4/web/node_modules/slickgrid'
 @ ./node_modules/slickgrid/slick.dataview.js 3:12-32
 @ ./pgadmin/static/bundle/slickgrid.js

ERROR in ./node_modules/slickgrid/slick.dataview.js
Module not found: Error: Can't resolve 'jquery.event.drag' in '/Users/dpage/git/pgadmin4/web/node_modules/slickgrid'
 @ ./node_modules/slickgrid/slick.dataview.js 6:20-48
 @ ./pgadmin/static/bundle/slickgrid.js

ERROR in ./node_modules/slickgrid/slick.dataview.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/node_modules/slickgrid'
 @ ./node_modules/slickgrid/slick.dataview.js 1:0-17
 @ ./pgadmin/static/bundle/slickgrid.js

ERROR in ./node_modules/slickgrid/slick.editors.js
Module not found: Error: Can't resolve 'jquery.ui' in '/Users/dpage/git/pgadmin4/web/node_modules/slickgrid'
 @ ./node_modules/slickgrid/slick.editors.js 3:12-32
 @ ./pgadmin/static/bundle/slickgrid.js

ERROR in ./node_modules/slickgrid/slick.editors.js
Module not found: Error: Can't resolve 'jquery.event.drag' in '/Users/dpage/git/pgadmin4/web/node_modules/slickgrid'
 @ ./node_modules/slickgrid/slick.editors.js 6:20-48
 @ ./pgadmin/static/bundle/slickgrid.js

ERROR in ./node_modules/slickgrid/slick.editors.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/node_modules/slickgrid'
 @ ./node_modules/slickgrid/slick.editors.js 1:0-17
 @ ./pgadmin/static/bundle/slickgrid.js

ERROR in ./node_modules/slickgrid/slick.formatters.js
Module not found: Error: Can't resolve 'jquery.ui' in '/Users/dpage/git/pgadmin4/web/node_modules/slickgrid'
 @ ./node_modules/slickgrid/slick.formatters.js 3:12-32
 @ ./pgadmin/static/bundle/slickgrid.js

ERROR in ./node_modules/slickgrid/slick.formatters.js
Module not found: Error: Can't resolve 'jquery.event.drag' in '/Users/dpage/git/pgadmin4/web/node_modules/slickgrid'
 @ ./node_modules/slickgrid/slick.formatters.js 6:20-48
 @ ./pgadmin/static/bundle/slickgrid.js

ERROR in ./node_modules/slickgrid/slick.formatters.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/node_modules/slickgrid'
 @ ./node_modules/slickgrid/slick.formatters.js 1:0-17
 @ ./pgadmin/static/bundle/slickgrid.js

ERROR in ./node_modules/slickgrid/lib/jquery.event.drag-2.3.0.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/node_modules/slickgrid/lib'
 @ ./node_modules/slickgrid/lib/jquery.event.drag-2.3.0.js 1:0-17
 @ ./pgadmin/static/bundle/slickgrid.js

ERROR in ./node_modules/slickgrid/plugins/slick.autotooltips.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/node_modules/slickgrid/plugins'
 @ ./node_modules/slickgrid/plugins/slick.autotooltips.js 1:0-17
 @ ./pgadmin/static/bundle/slickgrid.js

ERROR in ./node_modules/slickgrid/plugins/slick.cellrangedecorator.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/node_modules/slickgrid/plugins'
 @ ./node_modules/slickgrid/plugins/slick.cellrangedecorator.js 1:0-17
 @ ./pgadmin/static/bundle/slickgrid.js

ERROR in ./node_modules/slickgrid/plugins/slick.cellrangeselector.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/node_modules/slickgrid/plugins'
 @ ./node_modules/slickgrid/plugins/slick.cellrangeselector.js 1:0-17
 @ ./pgadmin/static/bundle/slickgrid.js

ERROR in ./regression/javascript/code_mirror_spec.jsx
Module not found: Error: Can't resolve '../../pgadmin/static/vendor/jquery/jquery-1.11.2' in '/Users/dpage/git/pgadmin4/web/regression/javascript'
 @ ./regression/javascript/code_mirror_spec.jsx 7:14-73

ERROR in ./regression/javascript/selection/grid_selector_spec.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/regression/javascript/selection'
 @ ./regression/javascript/selection/grid_selector_spec.js 3:14-31

ERROR in ./regression/javascript/selection/copy_data_spec.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/regression/javascript/selection'
 @ ./regression/javascript/selection/copy_data_spec.js 3:14-31

ERROR in ./regression/javascript/selection/column_selector_spec.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/regression/javascript/selection'
 @ ./regression/javascript/selection/column_selector_spec.js 3:14-31

ERROR in ./regression/javascript/selection/range_selection_helper_spec.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/regression/javascript/selection'
 @ ./regression/javascript/selection/range_selection_helper_spec.js 3:14-31

ERROR in ./regression/javascript/selection/row_selector_spec.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/regression/javascript/selection'
 @ ./regression/javascript/selection/row_selector_spec.js 3:14-31

ERROR in ./regression/javascript/selection/set_staged_rows_spec.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/regression/javascript/selection'
 @ ./regression/javascript/selection/set_staged_rows_spec.js 3:14-31

ERROR in ./regression/javascript/selection/xcell_selection_model_spec.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/regression/javascript/selection'
 @ ./regression/javascript/selection/xcell_selection_model_spec.js 15:14-31

ERROR in ./regression/javascript/slickgrid/cell_selector_spec.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/regression/javascript/slickgrid'
 @ ./regression/javascript/slickgrid/cell_selector_spec.js 3:14-31

ERROR in ./regression/javascript/slickgrid/event_handlers/handle_query_output_keyboard_event_spec.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/regression/javascript/slickgrid/event_handlers'
 @ ./regression/javascript/slickgrid/event_handlers/handle_query_output_keyboard_event_spec.js 25:14-31

ERROR in ./regression/javascript/alerts/alertify_wrapper_spec.js
Module not found: Error: Can't resolve '../../../pgadmin/static/vendor/alertifyjs/alertify' in '/Users/dpage/git/pgadmin4/web/regression/javascript/alerts'
 @ ./regression/javascript/alerts/alertify_wrapper_spec.js 3:16-77

ERROR in ./pgadmin/static/js/check_node_visibility.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/pgadmin/static/js'
 @ ./pgadmin/static/js/check_node_visibility.js 12:0-42:2
 @ ./regression/javascript/check_node_visibility_spec.js

ERROR in ./pgadmin/static/js/sqleditor_utils.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/pgadmin/static/js'
 @ ./pgadmin/static/js/sqleditor_utils.js 13:0-59:2
 @ ./regression/javascript/sqleditor_utils_spec.js

ERROR in ./pgadmin/static/js/alerts/alertify_wrapper.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/pgadmin/static/js/alerts'
 @ ./pgadmin/static/js/alerts/alertify_wrapper.js 3:0-39:2
 @ ./regression/javascript/alerts/alertify_wrapper_spec.js

ERROR in ./pgadmin/static/js/alerts/alertify_wrapper.js
Module not found: Error: Can't resolve 'alertify' in '/Users/dpage/git/pgadmin4/web/pgadmin/static/js/alerts'
 @ ./pgadmin/static/js/alerts/alertify_wrapper.js 3:0-39:2
 @ ./regression/javascript/alerts/alertify_wrapper_spec.js

ERROR in ./pgadmin/static/js/selection/active_cell_capture.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/pgadmin/static/js/selection'
 @ ./pgadmin/static/js/selection/active_cell_capture.js 3:0-185:2
 @ ./regression/javascript/selection/active_cell_capture_spec.js

ERROR in ./pgadmin/static/js/selection/grid_selector.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/pgadmin/static/js/selection'
 @ ./pgadmin/static/js/selection/grid_selector.js 3:0-76:2
 @ ./regression/javascript/selection/column_selector_spec.js

ERROR in ./pgadmin/static/js/selection/xcell_selection_model.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/pgadmin/static/js/selection'
 @ ./pgadmin/static/js/selection/xcell_selection_model.js 3:0-228:2
 @ ./regression/javascript/selection/grid_selector_spec.js

ERROR in ./pgadmin/static/js/selection/clipboard.js
Module not found: Error: Can't resolve 'alertify' in '/Users/dpage/git/pgadmin4/web/pgadmin/static/js/selection'
 @ ./pgadmin/static/js/selection/clipboard.js 3:0-77:2
 @ ./regression/javascript/selection/copy_data_spec.js

ERROR in ./pgadmin/static/js/selection/copy_data.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/pgadmin/static/js/selection'
 @ ./pgadmin/static/js/selection/copy_data.js 3:0-34:2
 @ ./regression/javascript/selection/copy_data_spec.js

ERROR in ./pgadmin/static/js/selection/column_selector.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/pgadmin/static/js/selection'
 @ ./pgadmin/static/js/selection/column_selector.js 3:0-117:2
 @ ./regression/javascript/selection/column_selector_spec.js

ERROR in ./pgadmin/static/js/selection/row_selector.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/pgadmin/static/js/selection'
 @ ./pgadmin/static/js/selection/row_selector.js 3:0-92:2
 @ ./regression/javascript/selection/row_selector_spec.js

ERROR in ./pgadmin/static/js/selection/set_staged_rows.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/pgadmin/static/js/selection'
 @ ./pgadmin/static/js/selection/set_staged_rows.js 12:0-99:2
 @ ./regression/javascript/selection/set_staged_rows_spec.js

ERROR in ./pgadmin/static/jsx/history/detail/code_mirror.jsx
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/pgadmin/static/jsx/history/detail'
 @ ./pgadmin/static/jsx/history/detail/code_mirror.jsx 13:14-31
 @ ./regression/javascript/code_mirror_spec.jsx

ERROR in ./pgadmin/browser/static/js/menu.js
Module not found: Error: Can't resolve 'jquery' in '/Users/dpage/git/pgadmin4/web/pgadmin/browser/static/js'
 @ ./pgadmin/browser/static/js/menu.js 5:0-424:2
 @ ./regression/javascript/browser/menu_spec.js

ERROR in ./pgadmin/static/js/check_node_visibility.js
Module not found: Error: Can't resolve 'underscore.string' in '/Users/dpage/git/pgadmin4/web/pgadmin/static/js'
 @ ./pgadmin/static/js/check_node_visibility.js 12:0-42:2
 @ ./regression/javascript/check_node_visibility_spec.js

ERROR in ./pgadmin/browser/static/js/menu.js
Module not found: Error: Can't resolve 'underscore.string' in '/Users/dpage/git/pgadmin4/web/pgadmin/browser/static/js'
 @ ./pgadmin/browser/static/js/menu.js 5:0-424:2
 @ ./regression/javascript/browser/menu_spec.js

ERROR in ./pgadmin/static/bundle/codemirror.js
Module not found: Error: Can't resolve 'pgadmin.sqlfoldcode' in '/Users/dpage/git/pgadmin4/web/pgadmin/static/bundle'
 @ ./pgadmin/static/bundle/codemirror.js 39:0-30
 @ ./pgadmin/static/jsx/history/detail/code_mirror.jsx
 @ ./regression/javascript/code_mirror_spec.jsx
webpack: Failed to compile.
PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "../../pgadmin/static/vendor/jquery/jquery-1.11.2"
  at regression/javascript/code_mirror_spec.jsx:83268

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "../../pgadmin/static/vendor/jquery/jquery-1.11.2"
  at regression/javascript/code_mirror_spec.jsx:83268

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/history/query_history_spec.jsx:50644

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/history/query_history_spec.jsx:50644

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "../../../pgadmin/static/vendor/alertifyjs/alertify"
  at regression/javascript/alerts/alertify_wrapper_spec.js:74

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "../../../pgadmin/static/vendor/alertifyjs/alertify"
  at regression/javascript/alerts/alertify_wrapper_spec.js:74

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "underscore.string"
  at regression/javascript/browser/menu_spec.js:1852

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "underscore.string"
  at regression/javascript/browser/menu_spec.js:1852

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/check_node_visibility_spec.js:1678

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/check_node_visibility_spec.js:1678

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/selection/active_cell_capture_spec.js:2318

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/selection/active_cell_capture_spec.js:2318

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/selection/column_selector_spec.js:7027

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/selection/column_selector_spec.js:7027

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/selection/copy_data_spec.js:6711

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/selection/copy_data_spec.js:6711

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/selection/grid_selector_spec.js:6834

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/selection/grid_selector_spec.js:6834

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/selection/range_boundary_navigator_spec.js:584

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/selection/range_boundary_navigator_spec.js:584

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/selection/range_selection_helper_spec.js:6198

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/selection/range_selection_helper_spec.js:6198

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/selection/row_selector_spec.js:6727

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/selection/row_selector_spec.js:6727

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/selection/set_staged_rows_spec.js:6198

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/selection/set_staged_rows_spec.js:6198

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/selection/xcell_selection_model_spec.js:7089

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/selection/xcell_selection_model_spec.js:7089

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/slickgrid/cell_selector_spec.js:6434

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/slickgrid/cell_selector_spec.js:6434

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/slickgrid/event_handlers/handle_query_output_keyboard_event_spec.js:6411

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/slickgrid/event_handlers/handle_query_output_keyboard_event_spec.js:6411

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/sqleditor_utils_spec.js:125

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at regression/javascript/sqleditor_utils_spec.js:125

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at pgadmin/static/bundle/slickgrid.js:1008

PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "jquery"
  at pgadmin/static/bundle/slickgrid.js:1008

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Re: [pgAdmin4]: Webpacking of static JS/CSS

From
Surinder Kumar
Date:
Hi

I'm currently working on first TODO: "Automatically handle static and template JS files"

As discussed with Ashesh, currently the paths to module id are written manually in webpack.config.js, instead the path defined in moudle's `def get_own_javascript()`  should be used.

So, we will be generating a paths.json file which will contain:

1. resolve > alias - path with reference to module id.(Static files)

2. externals - list of modules to be loaded dynamically on demand(Template files)

3. Shim module dependency

4. List of JS modules to be loaded in specified order.

Implementation:

To generate `paths.json` file, we will be using `Flask's test_client` to make an http request internally within the app context so we can call `current_app.javascripts` property and return the list of JS paths and write those into paths.json file and then use it in webpack.shim.js before the execution of `yarn run bundle` in `javascript_bundler.py`

For example:
@app.route('/get_script_paths')
def get_script_paths():
    from flask import current_app
    from pgadmin.utils.ajax import make_json_response

    return make_json_response(data=current_app.javascripts)

if config.DEBUG:
    with app.test_client() as client:
        import simplejson as json
        list_scripts = client.get('/get_script_paths')
        scripts = json.loads(list_scripts.data)

    javascriptBundler = JavascriptBundler()
    javascriptBundler.bundle(scripts['data'])

This also needs little change in module dependency we defined using 'When': 'node_name' in `def get_own_javascripts(...)` method
the module specified(name: module_name) is loaded when module given in `When` is expanded in node. Since we are using Webpack in which behaviour to load module is little different.

Now in webpack we are using `imports-loader` to load specific modules. So this is how it should work.

1. First load all modules which do not have dependency on any node like 'about', 'dashboard',  'server-group', 'server' etc.

2. Load module such as `Databases` node first before its child nodes are loaded.
Similarly load `Schemas` node before its child nodes are loaded as they are dependent on parent node.

Thanks,
Surinder
On Wed, Jul 5, 2017 at 8:22 PM, Sarah McAlear <smcalear@pivotal.io> wrote:
Hello,


​Things to discuss:

How to differentiate between a static and template JS
​​
.

What is the advantage of webpacking templated JS? It seems as though this creates a system in which the bundled dependencies have to refer back to the backend to load the templates. 
​Templated JS will not be part of generated bundle JS, they will load externally( an extra request will be made to server For example: translations.js)

If there is a performance win in packing templated JS then looking at it makes sense.  Otherwise it may make sense to put off until it is clear that the templated files should be dealt with by either de-templating them or bundling them where there is a clear reason.
​Template JS cannot be bundled, so i extract the <Jinja> code from template files and put into a separate file - ABC.js (also moved template files to static directory) and then load ABC.js dynamically as dependency of other modules.

However, we're wondering about possible performance penalties with templating larger files (as opposed to templating on-demand.) Since jinja templates can execute arbitrary python, this could get time expensive and further slow things like initial page-load.
Another concern is: what happens when a template gets out of date (e.g. if browser.js had previously filled in the content for 'panel_item.content' and had been cached, would it render a new version with the new values when needed? Or is it possible that we would get old content?) 
​That file will always gets new content when loaded dynamically, the content is not cached.​
 
Taks remaining:

​1. ​
Fix local variables which are declared without using var, have to check in each file
​ by​
 running eslint (For now, i will fix only errors which are giving error in browser).

​2. ​
Move non-template files from ’templates’ to ’static’ directory. List of
​ pending​
 modules is here: 
  • Tools (mostly all modules - 9 modules) 
  • Browser nodes - 3 modules(resource group, roles, tablespace)
  • ​About
    ​​
Also can we move 
​'​
dashboard, statistic
​s​
, preferences and help
​'​
 modules inside misc to preserve modularity as pgAdmin is modular
​ ?​

Is there anything from a organization stance you discussed in the previous email that needs to be done to make this usable and consistent?
​No​


Thanks,

George & Sarah 

Re: [pgAdmin4]: Webpacking of static JS/CSS

From
Surinder Kumar
Date:
​Hi Dave,

I didn't removed the vendor modules when i ran regression test cases, so modules were being referenced from vendor dir and passed for me.
Now I have fixed path references and test cases are working.

Please find attached patch.

Thanks
Surinder​

On Mon, Jul 17, 2017 at 3:18 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

I'm currently working on first TODO: "Automatically handle static and template JS files"

As discussed with Ashesh, currently the paths to module id are written manually in webpack.config.js, instead the path defined in moudle's `def get_own_javascript()`  should be used.

So, we will be generating a paths.json file which will contain:

1. resolve > alias - path with reference to module id.(Static files)

2. externals - list of modules to be loaded dynamically on demand(Template files)

3. Shim module dependency

4. List of JS modules to be loaded in specified order.

Implementation:

To generate `paths.json` file, we will be using `Flask's test_client` to make an http request internally within the app context so we can call `current_app.javascripts` property and return the list of JS paths and write those into paths.json file and then use it in webpack.shim.js before the execution of `yarn run bundle` in `javascript_bundler.py`

For example:
@app.route('/get_script_paths')
def get_script_paths():
    from flask import current_app
    from pgadmin.utils.ajax import make_json_response

    return make_json_response(data=current_app.javascripts)

if config.DEBUG:
    with app.test_client() as client:
        import simplejson as json
        list_scripts = client.get('/get_script_paths')
        scripts = json.loads(list_scripts.data)

    javascriptBundler = JavascriptBundler()
    javascriptBundler.bundle(scripts['data'])

This also needs little change in module dependency we defined using 'When': 'node_name' in `def get_own_javascripts(...)` method
the module specified(name: module_name) is loaded when module given in `When` is expanded in node. Since we are using Webpack in which behaviour to load module is little different.

Now in webpack we are using `imports-loader` to load specific modules. So this is how it should work.

1. First load all modules which do not have dependency on any node like 'about', 'dashboard',  'server-group', 'server' etc.

2. Load module such as `Databases` node first before its child nodes are loaded.
Similarly load `Schemas` node before its child nodes are loaded as they are dependent on parent node.

Thanks,
Surinder
On Wed, Jul 5, 2017 at 8:22 PM, Sarah McAlear <smcalear@pivotal.io> wrote:
Hello,


​Things to discuss:

How to differentiate between a static and template JS
​​
.

What is the advantage of webpacking templated JS? It seems as though this creates a system in which the bundled dependencies have to refer back to the backend to load the templates. 
​Templated JS will not be part of generated bundle JS, they will load externally( an extra request will be made to server For example: translations.js)

If there is a performance win in packing templated JS then looking at it makes sense.  Otherwise it may make sense to put off until it is clear that the templated files should be dealt with by either de-templating them or bundling them where there is a clear reason.
​Template JS cannot be bundled, so i extract the <Jinja> code from template files and put into a separate file - ABC.js (also moved template files to static directory) and then load ABC.js dynamically as dependency of other modules.

However, we're wondering about possible performance penalties with templating larger files (as opposed to templating on-demand.) Since jinja templates can execute arbitrary python, this could get time expensive and further slow things like initial page-load.
Another concern is: what happens when a template gets out of date (e.g. if browser.js had previously filled in the content for 'panel_item.content' and had been cached, would it render a new version with the new values when needed? Or is it possible that we would get old content?) 
​That file will always gets new content when loaded dynamically, the content is not cached.​
 
Taks remaining:

​1. ​
Fix local variables which are declared without using var, have to check in each file
​ by​
 running eslint (For now, i will fix only errors which are giving error in browser).

​2. ​
Move non-template files from ’templates’ to ’static’ directory. List of
​ pending​
 modules is here: 
  • Tools (mostly all modules - 9 modules) 
  • Browser nodes - 3 modules(resource group, roles, tablespace)
  • ​About
    ​​
Also can we move 
​'​
dashboard, statistic
​s​
, preferences and help
​'​
 modules inside misc to preserve modularity as pgAdmin is modular
​ ?​

Is there anything from a organization stance you discussed in the previous email that needs to be done to make this usable and consistent?
​No​


Thanks,

George & Sarah 


Attachment

Re: [pgAdmin4]: Webpacking of static JS/CSS

From
Dave Page
Date:
Hi

No errors now, but do you know why JS tests are being skipped?

PhantomJS 2.1.1 (Mac OS X 0.0.0): Executed 4 of 216 (skipped 212) SUCCESS (0.085 secs / 0.046 secs)

Thanks!

On Mon, Jul 17, 2017 at 12:07 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
​Hi Dave,

I didn't removed the vendor modules when i ran regression test cases, so modules were being referenced from vendor dir and passed for me.
Now I have fixed path references and test cases are working.

Please find attached patch.

Thanks
Surinder​

On Mon, Jul 17, 2017 at 3:18 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

I'm currently working on first TODO: "Automatically handle static and template JS files"

As discussed with Ashesh, currently the paths to module id are written manually in webpack.config.js, instead the path defined in moudle's `def get_own_javascript()`  should be used.

So, we will be generating a paths.json file which will contain:

1. resolve > alias - path with reference to module id.(Static files)

2. externals - list of modules to be loaded dynamically on demand(Template files)

3. Shim module dependency

4. List of JS modules to be loaded in specified order.

Implementation:

To generate `paths.json` file, we will be using `Flask's test_client` to make an http request internally within the app context so we can call `current_app.javascripts` property and return the list of JS paths and write those into paths.json file and then use it in webpack.shim.js before the execution of `yarn run bundle` in `javascript_bundler.py`

For example:
@app.route('/get_script_paths')
def get_script_paths():
    from flask import current_app
    from pgadmin.utils.ajax import make_json_response

    return make_json_response(data=current_app.javascripts)

if config.DEBUG:
    with app.test_client() as client:
        import simplejson as json
        list_scripts = client.get('/get_script_paths')
        scripts = json.loads(list_scripts.data)

    javascriptBundler = JavascriptBundler()
    javascriptBundler.bundle(scripts['data'])

This also needs little change in module dependency we defined using 'When': 'node_name' in `def get_own_javascripts(...)` method
the module specified(name: module_name) is loaded when module given in `When` is expanded in node. Since we are using Webpack in which behaviour to load module is little different.

Now in webpack we are using `imports-loader` to load specific modules. So this is how it should work.

1. First load all modules which do not have dependency on any node like 'about', 'dashboard',  'server-group', 'server' etc.

2. Load module such as `Databases` node first before its child nodes are loaded.
Similarly load `Schemas` node before its child nodes are loaded as they are dependent on parent node.

Thanks,
Surinder
On Wed, Jul 5, 2017 at 8:22 PM, Sarah McAlear <smcalear@pivotal.io> wrote:
Hello,


​Things to discuss:

How to differentiate between a static and template JS
​​
.

What is the advantage of webpacking templated JS? It seems as though this creates a system in which the bundled dependencies have to refer back to the backend to load the templates. 
​Templated JS will not be part of generated bundle JS, they will load externally( an extra request will be made to server For example: translations.js)

If there is a performance win in packing templated JS then looking at it makes sense.  Otherwise it may make sense to put off until it is clear that the templated files should be dealt with by either de-templating them or bundling them where there is a clear reason.
​Template JS cannot be bundled, so i extract the <Jinja> code from template files and put into a separate file - ABC.js (also moved template files to static directory) and then load ABC.js dynamically as dependency of other modules.

However, we're wondering about possible performance penalties with templating larger files (as opposed to templating on-demand.) Since jinja templates can execute arbitrary python, this could get time expensive and further slow things like initial page-load.
Another concern is: what happens when a template gets out of date (e.g. if browser.js had previously filled in the content for 'panel_item.content' and had been cached, would it render a new version with the new values when needed? Or is it possible that we would get old content?) 
​That file will always gets new content when loaded dynamically, the content is not cached.​
 
Taks remaining:

​1. ​
Fix local variables which are declared without using var, have to check in each file
​ by​
 running eslint (For now, i will fix only errors which are giving error in browser).

​2. ​
Move non-template files from ’templates’ to ’static’ directory. List of
​ pending​
 modules is here: 
  • Tools (mostly all modules - 9 modules) 
  • Browser nodes - 3 modules(resource group, roles, tablespace)
  • ​About
    ​​
Also can we move 
​'​
dashboard, statistic
​s​
, preferences and help
​'​
 modules inside misc to preserve modularity as pgAdmin is modular
​ ?​

Is there anything from a organization stance you discussed in the previous email that needs to be done to make this usable and consistent?
​No​


Thanks,

George & Sarah 





--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Re: [pgAdmin4]: Webpacking of static JS/CSS

From
Surinder Kumar
Date:
On Mon, Jul 17, 2017 at 4:52 PM, Dave Page <dpage@pgadmin.org> wrote:
Hi

No errors now, but do you know why JS tests are being skipped?
​No errors/warning in console even after settings `logLevel: config.LOG_DEBUG`. I am still debugging.

PhantomJS 2.1.1 (Mac OS X 0.0.0): Executed 4 of 216 (skipped 212) SUCCESS (0.085 secs / 0.046 secs)

Thanks!

On Mon, Jul 17, 2017 at 12:07 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
​Hi Dave,

I didn't removed the vendor modules when i ran regression test cases, so modules were being referenced from vendor dir and passed for me.
Now I have fixed path references and test cases are working.

Please find attached patch.

Thanks
Surinder​

On Mon, Jul 17, 2017 at 3:18 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

I'm currently working on first TODO: "Automatically handle static and template JS files"

As discussed with Ashesh, currently the paths to module id are written manually in webpack.config.js, instead the path defined in moudle's `def get_own_javascript()`  should be used.

So, we will be generating a paths.json file which will contain:

1. resolve > alias - path with reference to module id.(Static files)

2. externals - list of modules to be loaded dynamically on demand(Template files)

3. Shim module dependency

4. List of JS modules to be loaded in specified order.

Implementation:

To generate `paths.json` file, we will be using `Flask's test_client` to make an http request internally within the app context so we can call `current_app.javascripts` property and return the list of JS paths and write those into paths.json file and then use it in webpack.shim.js before the execution of `yarn run bundle` in `javascript_bundler.py`

For example:
@app.route('/get_script_paths')
def get_script_paths():
    from flask import current_app
    from pgadmin.utils.ajax import make_json_response

    return make_json_response(data=current_app.javascripts)

if config.DEBUG:
    with app.test_client() as client:
        import simplejson as json
        list_scripts = client.get('/get_script_paths')
        scripts = json.loads(list_scripts.data)

    javascriptBundler = JavascriptBundler()
    javascriptBundler.bundle(scripts['data'])

This also needs little change in module dependency we defined using 'When': 'node_name' in `def get_own_javascripts(...)` method
the module specified(name: module_name) is loaded when module given in `When` is expanded in node. Since we are using Webpack in which behaviour to load module is little different.

Now in webpack we are using `imports-loader` to load specific modules. So this is how it should work.

1. First load all modules which do not have dependency on any node like 'about', 'dashboard',  'server-group', 'server' etc.

2. Load module such as `Databases` node first before its child nodes are loaded.
Similarly load `Schemas` node before its child nodes are loaded as they are dependent on parent node.

Thanks,
Surinder
On Wed, Jul 5, 2017 at 8:22 PM, Sarah McAlear <smcalear@pivotal.io> wrote:
Hello,


​Things to discuss:

How to differentiate between a static and template JS
​​
.

What is the advantage of webpacking templated JS? It seems as though this creates a system in which the bundled dependencies have to refer back to the backend to load the templates. 
​Templated JS will not be part of generated bundle JS, they will load externally( an extra request will be made to server For example: translations.js)

If there is a performance win in packing templated JS then looking at it makes sense.  Otherwise it may make sense to put off until it is clear that the templated files should be dealt with by either de-templating them or bundling them where there is a clear reason.
​Template JS cannot be bundled, so i extract the <Jinja> code from template files and put into a separate file - ABC.js (also moved template files to static directory) and then load ABC.js dynamically as dependency of other modules.

However, we're wondering about possible performance penalties with templating larger files (as opposed to templating on-demand.) Since jinja templates can execute arbitrary python, this could get time expensive and further slow things like initial page-load.
Another concern is: what happens when a template gets out of date (e.g. if browser.js had previously filled in the content for 'panel_item.content' and had been cached, would it render a new version with the new values when needed? Or is it possible that we would get old content?) 
​That file will always gets new content when loaded dynamically, the content is not cached.​
 
Taks remaining:

​1. ​
Fix local variables which are declared without using var, have to check in each file
​ by​
 running eslint (For now, i will fix only errors which are giving error in browser).

​2. ​
Move non-template files from ’templates’ to ’static’ directory. List of
​ pending​
 modules is here: 
  • Tools (mostly all modules - 9 modules) 
  • Browser nodes - 3 modules(resource group, roles, tablespace)
  • ​About
    ​​
Also can we move 
​'​
dashboard, statistic
​s​
, preferences and help
​'​
 modules inside misc to preserve modularity as pgAdmin is modular
​ ?​

Is there anything from a organization stance you discussed in the previous email that needs to be done to make this usable and consistent?
​No​


Thanks,

George & Sarah 





--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Re: [pgAdmin4]: Webpacking of static JS/CSS

From
Surinder Kumar
Date:
Hi

Now all test cases are executing.
Please find updated patch.

Thanks
Surinder

On Mon, Jul 17, 2017 at 6:57 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
On Mon, Jul 17, 2017 at 4:52 PM, Dave Page <dpage@pgadmin.org> wrote:
Hi

No errors now, but do you know why JS tests are being skipped?
​No errors/warning in console even after settings `logLevel: config.LOG_DEBUG`. I am still debugging.

PhantomJS 2.1.1 (Mac OS X 0.0.0): Executed 4 of 216 (skipped 212) SUCCESS (0.085 secs / 0.046 secs)

Thanks!

On Mon, Jul 17, 2017 at 12:07 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
​Hi Dave,

I didn't removed the vendor modules when i ran regression test cases, so modules were being referenced from vendor dir and passed for me.
Now I have fixed path references and test cases are working.

Please find attached patch.

Thanks
Surinder​

On Mon, Jul 17, 2017 at 3:18 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

I'm currently working on first TODO: "Automatically handle static and template JS files"

As discussed with Ashesh, currently the paths to module id are written manually in webpack.config.js, instead the path defined in moudle's `def get_own_javascript()`  should be used.

So, we will be generating a paths.json file which will contain:

1. resolve > alias - path with reference to module id.(Static files)

2. externals - list of modules to be loaded dynamically on demand(Template files)

3. Shim module dependency

4. List of JS modules to be loaded in specified order.

Implementation:

To generate `paths.json` file, we will be using `Flask's test_client` to make an http request internally within the app context so we can call `current_app.javascripts` property and return the list of JS paths and write those into paths.json file and then use it in webpack.shim.js before the execution of `yarn run bundle` in `javascript_bundler.py`

For example:
@app.route('/get_script_paths')
def get_script_paths():
    from flask import current_app
    from pgadmin.utils.ajax import make_json_response

    return make_json_response(data=current_app.javascripts)

if config.DEBUG:
    with app.test_client() as client:
        import simplejson as json
        list_scripts = client.get('/get_script_paths')
        scripts = json.loads(list_scripts.data)

    javascriptBundler = JavascriptBundler()
    javascriptBundler.bundle(scripts['data'])

This also needs little change in module dependency we defined using 'When': 'node_name' in `def get_own_javascripts(...)` method
the module specified(name: module_name) is loaded when module given in `When` is expanded in node. Since we are using Webpack in which behaviour to load module is little different.

Now in webpack we are using `imports-loader` to load specific modules. So this is how it should work.

1. First load all modules which do not have dependency on any node like 'about', 'dashboard',  'server-group', 'server' etc.

2. Load module such as `Databases` node first before its child nodes are loaded.
Similarly load `Schemas` node before its child nodes are loaded as they are dependent on parent node.

Thanks,
Surinder
On Wed, Jul 5, 2017 at 8:22 PM, Sarah McAlear <smcalear@pivotal.io> wrote:
Hello,


​Things to discuss:

How to differentiate between a static and template JS
​​
.

What is the advantage of webpacking templated JS? It seems as though this creates a system in which the bundled dependencies have to refer back to the backend to load the templates. 
​Templated JS will not be part of generated bundle JS, they will load externally( an extra request will be made to server For example: translations.js)

If there is a performance win in packing templated JS then looking at it makes sense.  Otherwise it may make sense to put off until it is clear that the templated files should be dealt with by either de-templating them or bundling them where there is a clear reason.
​Template JS cannot be bundled, so i extract the <Jinja> code from template files and put into a separate file - ABC.js (also moved template files to static directory) and then load ABC.js dynamically as dependency of other modules.

However, we're wondering about possible performance penalties with templating larger files (as opposed to templating on-demand.) Since jinja templates can execute arbitrary python, this could get time expensive and further slow things like initial page-load.
Another concern is: what happens when a template gets out of date (e.g. if browser.js had previously filled in the content for 'panel_item.content' and had been cached, would it render a new version with the new values when needed? Or is it possible that we would get old content?) 
​That file will always gets new content when loaded dynamically, the content is not cached.​
 
Taks remaining:

​1. ​
Fix local variables which are declared without using var, have to check in each file
​ by​
 running eslint (For now, i will fix only errors which are giving error in browser).

​2. ​
Move non-template files from ’templates’ to ’static’ directory. List of
​ pending​
 modules is here: 
  • Tools (mostly all modules - 9 modules) 
  • Browser nodes - 3 modules(resource group, roles, tablespace)
  • ​About
    ​​
Also can we move 
​'​
dashboard, statistic
​s​
, preferences and help
​'​
 modules inside misc to preserve modularity as pgAdmin is modular
​ ?​

Is there anything from a organization stance you discussed in the previous email that needs to be done to make this usable and consistent?
​No​


Thanks,

George & Sarah 





--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


Attachment

Re: [pgAdmin4]: Webpacking of static JS/CSS

From
Dave Page
Date:
Thanks - applied!

Awesome work - on an average of 3 tests on my Mac, load time reduced from 11.55s with v1.6 to 5.53s with GIT Head.

On Mon, Jul 17, 2017 at 5:57 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

Now all test cases are executing.
Please find updated patch.

Thanks
Surinder

On Mon, Jul 17, 2017 at 6:57 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
On Mon, Jul 17, 2017 at 4:52 PM, Dave Page <dpage@pgadmin.org> wrote:
Hi

No errors now, but do you know why JS tests are being skipped?
​No errors/warning in console even after settings `logLevel: config.LOG_DEBUG`. I am still debugging.

PhantomJS 2.1.1 (Mac OS X 0.0.0): Executed 4 of 216 (skipped 212) SUCCESS (0.085 secs / 0.046 secs)

Thanks!

On Mon, Jul 17, 2017 at 12:07 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
​Hi Dave,

I didn't removed the vendor modules when i ran regression test cases, so modules were being referenced from vendor dir and passed for me.
Now I have fixed path references and test cases are working.

Please find attached patch.

Thanks
Surinder​

On Mon, Jul 17, 2017 at 3:18 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

I'm currently working on first TODO: "Automatically handle static and template JS files"

As discussed with Ashesh, currently the paths to module id are written manually in webpack.config.js, instead the path defined in moudle's `def get_own_javascript()`  should be used.

So, we will be generating a paths.json file which will contain:

1. resolve > alias - path with reference to module id.(Static files)

2. externals - list of modules to be loaded dynamically on demand(Template files)

3. Shim module dependency

4. List of JS modules to be loaded in specified order.

Implementation:

To generate `paths.json` file, we will be using `Flask's test_client` to make an http request internally within the app context so we can call `current_app.javascripts` property and return the list of JS paths and write those into paths.json file and then use it in webpack.shim.js before the execution of `yarn run bundle` in `javascript_bundler.py`

For example:
@app.route('/get_script_paths')
def get_script_paths():
    from flask import current_app
    from pgadmin.utils.ajax import make_json_response

    return make_json_response(data=current_app.javascripts)

if config.DEBUG:
    with app.test_client() as client:
        import simplejson as json
        list_scripts = client.get('/get_script_paths')
        scripts = json.loads(list_scripts.data)

    javascriptBundler = JavascriptBundler()
    javascriptBundler.bundle(scripts['data'])

This also needs little change in module dependency we defined using 'When': 'node_name' in `def get_own_javascripts(...)` method
the module specified(name: module_name) is loaded when module given in `When` is expanded in node. Since we are using Webpack in which behaviour to load module is little different.

Now in webpack we are using `imports-loader` to load specific modules. So this is how it should work.

1. First load all modules which do not have dependency on any node like 'about', 'dashboard',  'server-group', 'server' etc.

2. Load module such as `Databases` node first before its child nodes are loaded.
Similarly load `Schemas` node before its child nodes are loaded as they are dependent on parent node.

Thanks,
Surinder
On Wed, Jul 5, 2017 at 8:22 PM, Sarah McAlear <smcalear@pivotal.io> wrote:
Hello,


​Things to discuss:

How to differentiate between a static and template JS
​​
.

What is the advantage of webpacking templated JS? It seems as though this creates a system in which the bundled dependencies have to refer back to the backend to load the templates. 
​Templated JS will not be part of generated bundle JS, they will load externally( an extra request will be made to server For example: translations.js)

If there is a performance win in packing templated JS then looking at it makes sense.  Otherwise it may make sense to put off until it is clear that the templated files should be dealt with by either de-templating them or bundling them where there is a clear reason.
​Template JS cannot be bundled, so i extract the <Jinja> code from template files and put into a separate file - ABC.js (also moved template files to static directory) and then load ABC.js dynamically as dependency of other modules.

However, we're wondering about possible performance penalties with templating larger files (as opposed to templating on-demand.) Since jinja templates can execute arbitrary python, this could get time expensive and further slow things like initial page-load.
Another concern is: what happens when a template gets out of date (e.g. if browser.js had previously filled in the content for 'panel_item.content' and had been cached, would it render a new version with the new values when needed? Or is it possible that we would get old content?) 
​That file will always gets new content when loaded dynamically, the content is not cached.​
 
Taks remaining:

​1. ​
Fix local variables which are declared without using var, have to check in each file
​ by​
 running eslint (For now, i will fix only errors which are giving error in browser).

​2. ​
Move non-template files from ’templates’ to ’static’ directory. List of
​ pending​
 modules is here: 
  • Tools (mostly all modules - 9 modules) 
  • Browser nodes - 3 modules(resource group, roles, tablespace)
  • ​About
    ​​
Also can we move 
​'​
dashboard, statistic
​s​
, preferences and help
​'​
 modules inside misc to preserve modularity as pgAdmin is modular
​ ?​

Is there anything from a organization stance you discussed in the previous email that needs to be done to make this usable and consistent?
​No​


Thanks,

George & Sarah 





--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company





--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Re: [pgAdmin4]: Webpacking of static JS/CSS

From
Khushboo Vashi
Date:


On Tue, Jul 18, 2017 at 7:46 PM, Dave Page <dpage@pgadmin.org> wrote:
Thanks - applied!

Awesome work - on an average of 3 tests on my Mac, load time reduced from 11.55s with v1.6 to 5.53s with GIT Head.

Surinder, great work...
 
On Mon, Jul 17, 2017 at 5:57 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

Now all test cases are executing.
Please find updated patch.

Thanks
Surinder

On Mon, Jul 17, 2017 at 6:57 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
On Mon, Jul 17, 2017 at 4:52 PM, Dave Page <dpage@pgadmin.org> wrote:
Hi

No errors now, but do you know why JS tests are being skipped?
​No errors/warning in console even after settings `logLevel: config.LOG_DEBUG`. I am still debugging.

PhantomJS 2.1.1 (Mac OS X 0.0.0): Executed 4 of 216 (skipped 212) SUCCESS (0.085 secs / 0.046 secs)

Thanks!

On Mon, Jul 17, 2017 at 12:07 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
​Hi Dave,

I didn't removed the vendor modules when i ran regression test cases, so modules were being referenced from vendor dir and passed for me.
Now I have fixed path references and test cases are working.

Please find attached patch.

Thanks
Surinder​

On Mon, Jul 17, 2017 at 3:18 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

I'm currently working on first TODO: "Automatically handle static and template JS files"

As discussed with Ashesh, currently the paths to module id are written manually in webpack.config.js, instead the path defined in moudle's `def get_own_javascript()`  should be used.

So, we will be generating a paths.json file which will contain:

1. resolve > alias - path with reference to module id.(Static files)

2. externals - list of modules to be loaded dynamically on demand(Template files)

3. Shim module dependency

4. List of JS modules to be loaded in specified order.

Implementation:

To generate `paths.json` file, we will be using `Flask's test_client` to make an http request internally within the app context so we can call `current_app.javascripts` property and return the list of JS paths and write those into paths.json file and then use it in webpack.shim.js before the execution of `yarn run bundle` in `javascript_bundler.py`

For example:
@app.route('/get_script_paths')
def get_script_paths():
    from flask import current_app
    from pgadmin.utils.ajax import make_json_response

    return make_json_response(data=current_app.javascripts)

if config.DEBUG:
    with app.test_client() as client:
        import simplejson as json
        list_scripts = client.get('/get_script_paths')
        scripts = json.loads(list_scripts.data)

    javascriptBundler = JavascriptBundler()
    javascriptBundler.bundle(scripts['data'])

This also needs little change in module dependency we defined using 'When': 'node_name' in `def get_own_javascripts(...)` method
the module specified(name: module_name) is loaded when module given in `When` is expanded in node. Since we are using Webpack in which behaviour to load module is little different.

Now in webpack we are using `imports-loader` to load specific modules. So this is how it should work.

1. First load all modules which do not have dependency on any node like 'about', 'dashboard',  'server-group', 'server' etc.

2. Load module such as `Databases` node first before its child nodes are loaded.
Similarly load `Schemas` node before its child nodes are loaded as they are dependent on parent node.

Thanks,
Surinder
On Wed, Jul 5, 2017 at 8:22 PM, Sarah McAlear <smcalear@pivotal.io> wrote:
Hello,


​Things to discuss:

How to differentiate between a static and template JS
​​
.

What is the advantage of webpacking templated JS? It seems as though this creates a system in which the bundled dependencies have to refer back to the backend to load the templates. 
​Templated JS will not be part of generated bundle JS, they will load externally( an extra request will be made to server For example: translations.js)

If there is a performance win in packing templated JS then looking at it makes sense.  Otherwise it may make sense to put off until it is clear that the templated files should be dealt with by either de-templating them or bundling them where there is a clear reason.
​Template JS cannot be bundled, so i extract the <Jinja> code from template files and put into a separate file - ABC.js (also moved template files to static directory) and then load ABC.js dynamically as dependency of other modules.

However, we're wondering about possible performance penalties with templating larger files (as opposed to templating on-demand.) Since jinja templates can execute arbitrary python, this could get time expensive and further slow things like initial page-load.
Another concern is: what happens when a template gets out of date (e.g. if browser.js had previously filled in the content for 'panel_item.content' and had been cached, would it render a new version with the new values when needed? Or is it possible that we would get old content?) 
​That file will always gets new content when loaded dynamically, the content is not cached.​
 
Taks remaining:

​1. ​
Fix local variables which are declared without using var, have to check in each file
​ by​
 running eslint (For now, i will fix only errors which are giving error in browser).

​2. ​
Move non-template files from ’templates’ to ’static’ directory. List of
​ pending​
 modules is here: 
  • Tools (mostly all modules - 9 modules) 
  • Browser nodes - 3 modules(resource group, roles, tablespace)
  • ​About
    ​​
Also can we move 
​'​
dashboard, statistic
​s​
, preferences and help
​'​
 modules inside misc to preserve modularity as pgAdmin is modular
​ ?​

Is there anything from a organization stance you discussed in the previous email that needs to be done to make this usable and consistent?
​No​


Thanks,

George & Sarah 





--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company





--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Re: [pgAdmin4]: Webpacking of static JS/CSS

From
Surinder Kumar
Date:
Hi

1. As Slickgrid has dependency of `jQuery-ui`, it was missed. now added.
2. Column sorting for collection nodes sometimes failing when clicked on different collection nodes.

Please find attached patch.

Thanks
Surinder 

On Tue, Jul 18, 2017 at 8:20 PM, Khushboo Vashi <khushboo.vashi@enterprisedb.com> wrote:


On Tue, Jul 18, 2017 at 7:46 PM, Dave Page <dpage@pgadmin.org> wrote:
Thanks - applied!

Awesome work - on an average of 3 tests on my Mac, load time reduced from 11.55s with v1.6 to 5.53s with GIT Head.
​Thanks to all​

Surinder, great work...
 
On Mon, Jul 17, 2017 at 5:57 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

Now all test cases are executing.
Please find updated patch.

Thanks
Surinder

On Mon, Jul 17, 2017 at 6:57 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
On Mon, Jul 17, 2017 at 4:52 PM, Dave Page <dpage@pgadmin.org> wrote:
Hi

No errors now, but do you know why JS tests are being skipped?
​No errors/warning in console even after settings `logLevel: config.LOG_DEBUG`. I am still debugging.

PhantomJS 2.1.1 (Mac OS X 0.0.0): Executed 4 of 216 (skipped 212) SUCCESS (0.085 secs / 0.046 secs)

Thanks!

On Mon, Jul 17, 2017 at 12:07 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
​Hi Dave,

I didn't removed the vendor modules when i ran regression test cases, so modules were being referenced from vendor dir and passed for me.
Now I have fixed path references and test cases are working.

Please find attached patch.

Thanks
Surinder​

On Mon, Jul 17, 2017 at 3:18 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

I'm currently working on first TODO: "Automatically handle static and template JS files"

As discussed with Ashesh, currently the paths to module id are written manually in webpack.config.js, instead the path defined in moudle's `def get_own_javascript()`  should be used.

So, we will be generating a paths.json file which will contain:

1. resolve > alias - path with reference to module id.(Static files)

2. externals - list of modules to be loaded dynamically on demand(Template files)

3. Shim module dependency

4. List of JS modules to be loaded in specified order.

Implementation:

To generate `paths.json` file, we will be using `Flask's test_client` to make an http request internally within the app context so we can call `current_app.javascripts` property and return the list of JS paths and write those into paths.json file and then use it in webpack.shim.js before the execution of `yarn run bundle` in `javascript_bundler.py`

For example:
@app.route('/get_script_paths')
def get_script_paths():
    from flask import current_app
    from pgadmin.utils.ajax import make_json_response

    return make_json_response(data=current_app.javascripts)

if config.DEBUG:
    with app.test_client() as client:
        import simplejson as json
        list_scripts = client.get('/get_script_paths')
        scripts = json.loads(list_scripts.data)

    javascriptBundler = JavascriptBundler()
    javascriptBundler.bundle(scripts['data'])

This also needs little change in module dependency we defined using 'When': 'node_name' in `def get_own_javascripts(...)` method
the module specified(name: module_name) is loaded when module given in `When` is expanded in node. Since we are using Webpack in which behaviour to load module is little different.

Now in webpack we are using `imports-loader` to load specific modules. So this is how it should work.

1. First load all modules which do not have dependency on any node like 'about', 'dashboard',  'server-group', 'server' etc.

2. Load module such as `Databases` node first before its child nodes are loaded.
Similarly load `Schemas` node before its child nodes are loaded as they are dependent on parent node.

Thanks,
Surinder
On Wed, Jul 5, 2017 at 8:22 PM, Sarah McAlear <smcalear@pivotal.io> wrote:
Hello,


​Things to discuss:

How to differentiate between a static and template JS
​​
.

What is the advantage of webpacking templated JS? It seems as though this creates a system in which the bundled dependencies have to refer back to the backend to load the templates. 
​Templated JS will not be part of generated bundle JS, they will load externally( an extra request will be made to server For example: translations.js)

If there is a performance win in packing templated JS then looking at it makes sense.  Otherwise it may make sense to put off until it is clear that the templated files should be dealt with by either de-templating them or bundling them where there is a clear reason.
​Template JS cannot be bundled, so i extract the <Jinja> code from template files and put into a separate file - ABC.js (also moved template files to static directory) and then load ABC.js dynamically as dependency of other modules.

However, we're wondering about possible performance penalties with templating larger files (as opposed to templating on-demand.) Since jinja templates can execute arbitrary python, this could get time expensive and further slow things like initial page-load.
Another concern is: what happens when a template gets out of date (e.g. if browser.js had previously filled in the content for 'panel_item.content' and had been cached, would it render a new version with the new values when needed? Or is it possible that we would get old content?) 
​That file will always gets new content when loaded dynamically, the content is not cached.​
 
Taks remaining:

​1. ​
Fix local variables which are declared without using var, have to check in each file
​ by​
 running eslint (For now, i will fix only errors which are giving error in browser).

​2. ​
Move non-template files from ’templates’ to ’static’ directory. List of
​ pending​
 modules is here: 
  • Tools (mostly all modules - 9 modules) 
  • Browser nodes - 3 modules(resource group, roles, tablespace)
  • ​About
    ​​
Also can we move 
​'​
dashboard, statistic
​s​
, preferences and help
​'​
 modules inside misc to preserve modularity as pgAdmin is modular
​ ?​

Is there anything from a organization stance you discussed in the previous email that needs to be done to make this usable and consistent?
​No​


Thanks,

George & Sarah 





--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company





--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


Attachment

Re: [pgAdmin4]: Webpacking of static JS/CSS

From
Dave Page
Date:
Thanks, applied.

On Tue, Jul 18, 2017 at 4:12 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

1. As Slickgrid has dependency of `jQuery-ui`, it was missed. now added.
2. Column sorting for collection nodes sometimes failing when clicked on different collection nodes.

Please find attached patch.

Thanks
Surinder 

On Tue, Jul 18, 2017 at 8:20 PM, Khushboo Vashi <khushboo.vashi@enterprisedb.com> wrote:


On Tue, Jul 18, 2017 at 7:46 PM, Dave Page <dpage@pgadmin.org> wrote:
Thanks - applied!

Awesome work - on an average of 3 tests on my Mac, load time reduced from 11.55s with v1.6 to 5.53s with GIT Head.
​Thanks to all​

Surinder, great work...
 
On Mon, Jul 17, 2017 at 5:57 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

Now all test cases are executing.
Please find updated patch.

Thanks
Surinder

On Mon, Jul 17, 2017 at 6:57 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
On Mon, Jul 17, 2017 at 4:52 PM, Dave Page <dpage@pgadmin.org> wrote:
Hi

No errors now, but do you know why JS tests are being skipped?
​No errors/warning in console even after settings `logLevel: config.LOG_DEBUG`. I am still debugging.

PhantomJS 2.1.1 (Mac OS X 0.0.0): Executed 4 of 216 (skipped 212) SUCCESS (0.085 secs / 0.046 secs)

Thanks!

On Mon, Jul 17, 2017 at 12:07 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
​Hi Dave,

I didn't removed the vendor modules when i ran regression test cases, so modules were being referenced from vendor dir and passed for me.
Now I have fixed path references and test cases are working.

Please find attached patch.

Thanks
Surinder​

On Mon, Jul 17, 2017 at 3:18 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

I'm currently working on first TODO: "Automatically handle static and template JS files"

As discussed with Ashesh, currently the paths to module id are written manually in webpack.config.js, instead the path defined in moudle's `def get_own_javascript()`  should be used.

So, we will be generating a paths.json file which will contain:

1. resolve > alias - path with reference to module id.(Static files)

2. externals - list of modules to be loaded dynamically on demand(Template files)

3. Shim module dependency

4. List of JS modules to be loaded in specified order.

Implementation:

To generate `paths.json` file, we will be using `Flask's test_client` to make an http request internally within the app context so we can call `current_app.javascripts` property and return the list of JS paths and write those into paths.json file and then use it in webpack.shim.js before the execution of `yarn run bundle` in `javascript_bundler.py`

For example:
@app.route('/get_script_paths')
def get_script_paths():
    from flask import current_app
    from pgadmin.utils.ajax import make_json_response

    return make_json_response(data=current_app.javascripts)

if config.DEBUG:
    with app.test_client() as client:
        import simplejson as json
        list_scripts = client.get('/get_script_paths')
        scripts = json.loads(list_scripts.data)

    javascriptBundler = JavascriptBundler()
    javascriptBundler.bundle(scripts['data'])

This also needs little change in module dependency we defined using 'When': 'node_name' in `def get_own_javascripts(...)` method
the module specified(name: module_name) is loaded when module given in `When` is expanded in node. Since we are using Webpack in which behaviour to load module is little different.

Now in webpack we are using `imports-loader` to load specific modules. So this is how it should work.

1. First load all modules which do not have dependency on any node like 'about', 'dashboard',  'server-group', 'server' etc.

2. Load module such as `Databases` node first before its child nodes are loaded.
Similarly load `Schemas` node before its child nodes are loaded as they are dependent on parent node.

Thanks,
Surinder
On Wed, Jul 5, 2017 at 8:22 PM, Sarah McAlear <smcalear@pivotal.io> wrote:
Hello,


​Things to discuss:

How to differentiate between a static and template JS
​​
.

What is the advantage of webpacking templated JS? It seems as though this creates a system in which the bundled dependencies have to refer back to the backend to load the templates. 
​Templated JS will not be part of generated bundle JS, they will load externally( an extra request will be made to server For example: translations.js)

If there is a performance win in packing templated JS then looking at it makes sense.  Otherwise it may make sense to put off until it is clear that the templated files should be dealt with by either de-templating them or bundling them where there is a clear reason.
​Template JS cannot be bundled, so i extract the <Jinja> code from template files and put into a separate file - ABC.js (also moved template files to static directory) and then load ABC.js dynamically as dependency of other modules.

However, we're wondering about possible performance penalties with templating larger files (as opposed to templating on-demand.) Since jinja templates can execute arbitrary python, this could get time expensive and further slow things like initial page-load.
Another concern is: what happens when a template gets out of date (e.g. if browser.js had previously filled in the content for 'panel_item.content' and had been cached, would it render a new version with the new values when needed? Or is it possible that we would get old content?) 
​That file will always gets new content when loaded dynamically, the content is not cached.​
 
Taks remaining:

​1. ​
Fix local variables which are declared without using var, have to check in each file
​ by​
 running eslint (For now, i will fix only errors which are giving error in browser).

​2. ​
Move non-template files from ’templates’ to ’static’ directory. List of
​ pending​
 modules is here: 
  • Tools (mostly all modules - 9 modules) 
  • Browser nodes - 3 modules(resource group, roles, tablespace)
  • ​About
    ​​
Also can we move 
​'​
dashboard, statistic
​s​
, preferences and help
​'​
 modules inside misc to preserve modularity as pgAdmin is modular
​ ?​

Is there anything from a organization stance you discussed in the previous email that needs to be done to make this usable and consistent?
​No​


Thanks,

George & Sarah 





--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company





--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company





--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Re: [pgAdmin4]: Webpacking of static JS/CSS

From
Murtuza Zabuawala
Date:
Great job Surinder, Load time ~2 sec on browser :)
 
Inline image 1


--
Regards,
Murtuza Zabuawala
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Tue, Jul 18, 2017 at 9:01 PM, Dave Page <dpage@pgadmin.org> wrote:
Thanks, applied.

On Tue, Jul 18, 2017 at 4:12 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

1. As Slickgrid has dependency of `jQuery-ui`, it was missed. now added.
2. Column sorting for collection nodes sometimes failing when clicked on different collection nodes.

Please find attached patch.

Thanks
Surinder 

On Tue, Jul 18, 2017 at 8:20 PM, Khushboo Vashi <khushboo.vashi@enterprisedb.com> wrote:


On Tue, Jul 18, 2017 at 7:46 PM, Dave Page <dpage@pgadmin.org> wrote:
Thanks - applied!

Awesome work - on an average of 3 tests on my Mac, load time reduced from 11.55s with v1.6 to 5.53s with GIT Head.
​Thanks to all​

Surinder, great work...
 
On Mon, Jul 17, 2017 at 5:57 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

Now all test cases are executing.
Please find updated patch.

Thanks
Surinder

On Mon, Jul 17, 2017 at 6:57 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
On Mon, Jul 17, 2017 at 4:52 PM, Dave Page <dpage@pgadmin.org> wrote:
Hi

No errors now, but do you know why JS tests are being skipped?
​No errors/warning in console even after settings `logLevel: config.LOG_DEBUG`. I am still debugging.

PhantomJS 2.1.1 (Mac OS X 0.0.0): Executed 4 of 216 (skipped 212) SUCCESS (0.085 secs / 0.046 secs)

Thanks!

On Mon, Jul 17, 2017 at 12:07 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
​Hi Dave,

I didn't removed the vendor modules when i ran regression test cases, so modules were being referenced from vendor dir and passed for me.
Now I have fixed path references and test cases are working.

Please find attached patch.

Thanks
Surinder​

On Mon, Jul 17, 2017 at 3:18 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

I'm currently working on first TODO: "Automatically handle static and template JS files"

As discussed with Ashesh, currently the paths to module id are written manually in webpack.config.js, instead the path defined in moudle's `def get_own_javascript()`  should be used.

So, we will be generating a paths.json file which will contain:

1. resolve > alias - path with reference to module id.(Static files)

2. externals - list of modules to be loaded dynamically on demand(Template files)

3. Shim module dependency

4. List of JS modules to be loaded in specified order.

Implementation:

To generate `paths.json` file, we will be using `Flask's test_client` to make an http request internally within the app context so we can call `current_app.javascripts` property and return the list of JS paths and write those into paths.json file and then use it in webpack.shim.js before the execution of `yarn run bundle` in `javascript_bundler.py`

For example:
@app.route('/get_script_paths')
def get_script_paths():
    from flask import current_app
    from pgadmin.utils.ajax import make_json_response

    return make_json_response(data=current_app.javascripts)

if config.DEBUG:
    with app.test_client() as client:
        import simplejson as json
        list_scripts = client.get('/get_script_paths')
        scripts = json.loads(list_scripts.data)

    javascriptBundler = JavascriptBundler()
    javascriptBundler.bundle(scripts['data'])

This also needs little change in module dependency we defined using 'When': 'node_name' in `def get_own_javascripts(...)` method
the module specified(name: module_name) is loaded when module given in `When` is expanded in node. Since we are using Webpack in which behaviour to load module is little different.

Now in webpack we are using `imports-loader` to load specific modules. So this is how it should work.

1. First load all modules which do not have dependency on any node like 'about', 'dashboard',  'server-group', 'server' etc.

2. Load module such as `Databases` node first before its child nodes are loaded.
Similarly load `Schemas` node before its child nodes are loaded as they are dependent on parent node.

Thanks,
Surinder
On Wed, Jul 5, 2017 at 8:22 PM, Sarah McAlear <smcalear@pivotal.io> wrote:
Hello,


​Things to discuss:

How to differentiate between a static and template JS
​​
.

What is the advantage of webpacking templated JS? It seems as though this creates a system in which the bundled dependencies have to refer back to the backend to load the templates. 
​Templated JS will not be part of generated bundle JS, they will load externally( an extra request will be made to server For example: translations.js)

If there is a performance win in packing templated JS then looking at it makes sense.  Otherwise it may make sense to put off until it is clear that the templated files should be dealt with by either de-templating them or bundling them where there is a clear reason.
​Template JS cannot be bundled, so i extract the <Jinja> code from template files and put into a separate file - ABC.js (also moved template files to static directory) and then load ABC.js dynamically as dependency of other modules.

However, we're wondering about possible performance penalties with templating larger files (as opposed to templating on-demand.) Since jinja templates can execute arbitrary python, this could get time expensive and further slow things like initial page-load.
Another concern is: what happens when a template gets out of date (e.g. if browser.js had previously filled in the content for 'panel_item.content' and had been cached, would it render a new version with the new values when needed? Or is it possible that we would get old content?) 
​That file will always gets new content when loaded dynamically, the content is not cached.​
 
Taks remaining:

​1. ​
Fix local variables which are declared without using var, have to check in each file
​ by​
 running eslint (For now, i will fix only errors which are giving error in browser).

​2. ​
Move non-template files from ’templates’ to ’static’ directory. List of
​ pending​
 modules is here: 
  • Tools (mostly all modules - 9 modules) 
  • Browser nodes - 3 modules(resource group, roles, tablespace)
  • ​About
    ​​
Also can we move 
​'​
dashboard, statistic
​s​
, preferences and help
​'​
 modules inside misc to preserve modularity as pgAdmin is modular
​ ?​

Is there anything from a organization stance you discussed in the previous email that needs to be done to make this usable and consistent?
​No​


Thanks,

George & Sarah 





--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company





--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company





--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Re: [pgAdmin4]: Webpacking of static JS/CSS

From
Surinder Kumar
Date:
Hi

Fix library path reference for `jquery.contextmenu'. This issue was only reproducible on Linux machine.
So, i setup pgAdmin4 on Ubuntu VM and tested the patch and it works.

Please find attached patch.

Thanks,
Surinder

On Tue, Jul 18, 2017 at 9:05 PM, Murtuza Zabuawala <murtuza.zabuawala@enterprisedb.com> wrote:
Great job Surinder, Load time ~2 sec on browser :)
 
Inline image 1


--
Regards,
Murtuza Zabuawala
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Tue, Jul 18, 2017 at 9:01 PM, Dave Page <dpage@pgadmin.org> wrote:
Thanks, applied.

On Tue, Jul 18, 2017 at 4:12 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

1. As Slickgrid has dependency of `jQuery-ui`, it was missed. now added.
2. Column sorting for collection nodes sometimes failing when clicked on different collection nodes.

Please find attached patch.

Thanks
Surinder 

On Tue, Jul 18, 2017 at 8:20 PM, Khushboo Vashi <khushboo.vashi@enterprisedb.com> wrote:


On Tue, Jul 18, 2017 at 7:46 PM, Dave Page <dpage@pgadmin.org> wrote:
Thanks - applied!

Awesome work - on an average of 3 tests on my Mac, load time reduced from 11.55s with v1.6 to 5.53s with GIT Head.
​Thanks to all​

Surinder, great work...
 
On Mon, Jul 17, 2017 at 5:57 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

Now all test cases are executing.
Please find updated patch.

Thanks
Surinder

On Mon, Jul 17, 2017 at 6:57 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
On Mon, Jul 17, 2017 at 4:52 PM, Dave Page <dpage@pgadmin.org> wrote:
Hi

No errors now, but do you know why JS tests are being skipped?
​No errors/warning in console even after settings `logLevel: config.LOG_DEBUG`. I am still debugging.

PhantomJS 2.1.1 (Mac OS X 0.0.0): Executed 4 of 216 (skipped 212) SUCCESS (0.085 secs / 0.046 secs)

Thanks!

On Mon, Jul 17, 2017 at 12:07 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
​Hi Dave,

I didn't removed the vendor modules when i ran regression test cases, so modules were being referenced from vendor dir and passed for me.
Now I have fixed path references and test cases are working.

Please find attached patch.

Thanks
Surinder​

On Mon, Jul 17, 2017 at 3:18 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

I'm currently working on first TODO: "Automatically handle static and template JS files"

As discussed with Ashesh, currently the paths to module id are written manually in webpack.config.js, instead the path defined in moudle's `def get_own_javascript()`  should be used.

So, we will be generating a paths.json file which will contain:

1. resolve > alias - path with reference to module id.(Static files)

2. externals - list of modules to be loaded dynamically on demand(Template files)

3. Shim module dependency

4. List of JS modules to be loaded in specified order.

Implementation:

To generate `paths.json` file, we will be using `Flask's test_client` to make an http request internally within the app context so we can call `current_app.javascripts` property and return the list of JS paths and write those into paths.json file and then use it in webpack.shim.js before the execution of `yarn run bundle` in `javascript_bundler.py`

For example:
@app.route('/get_script_paths')
def get_script_paths():
    from flask import current_app
    from pgadmin.utils.ajax import make_json_response

    return make_json_response(data=current_app.javascripts)

if config.DEBUG:
    with app.test_client() as client:
        import simplejson as json
        list_scripts = client.get('/get_script_paths')
        scripts = json.loads(list_scripts.data)

    javascriptBundler = JavascriptBundler()
    javascriptBundler.bundle(scripts['data'])

This also needs little change in module dependency we defined using 'When': 'node_name' in `def get_own_javascripts(...)` method
the module specified(name: module_name) is loaded when module given in `When` is expanded in node. Since we are using Webpack in which behaviour to load module is little different.

Now in webpack we are using `imports-loader` to load specific modules. So this is how it should work.

1. First load all modules which do not have dependency on any node like 'about', 'dashboard',  'server-group', 'server' etc.

2. Load module such as `Databases` node first before its child nodes are loaded.
Similarly load `Schemas` node before its child nodes are loaded as they are dependent on parent node.

Thanks,
Surinder
On Wed, Jul 5, 2017 at 8:22 PM, Sarah McAlear <smcalear@pivotal.io> wrote:
Hello,


​Things to discuss:

How to differentiate between a static and template JS
​​
.

What is the advantage of webpacking templated JS? It seems as though this creates a system in which the bundled dependencies have to refer back to the backend to load the templates. 
​Templated JS will not be part of generated bundle JS, they will load externally( an extra request will be made to server For example: translations.js)

If there is a performance win in packing templated JS then looking at it makes sense.  Otherwise it may make sense to put off until it is clear that the templated files should be dealt with by either de-templating them or bundling them where there is a clear reason.
​Template JS cannot be bundled, so i extract the <Jinja> code from template files and put into a separate file - ABC.js (also moved template files to static directory) and then load ABC.js dynamically as dependency of other modules.

However, we're wondering about possible performance penalties with templating larger files (as opposed to templating on-demand.) Since jinja templates can execute arbitrary python, this could get time expensive and further slow things like initial page-load.
Another concern is: what happens when a template gets out of date (e.g. if browser.js had previously filled in the content for 'panel_item.content' and had been cached, would it render a new version with the new values when needed? Or is it possible that we would get old content?) 
​That file will always gets new content when loaded dynamically, the content is not cached.​
 
Taks remaining:

​1. ​
Fix local variables which are declared without using var, have to check in each file
​ by​
 running eslint (For now, i will fix only errors which are giving error in browser).

​2. ​
Move non-template files from ’templates’ to ’static’ directory. List of
​ pending​
 modules is here: 
  • Tools (mostly all modules - 9 modules) 
  • Browser nodes - 3 modules(resource group, roles, tablespace)
  • ​About
    ​​
Also can we move 
​'​
dashboard, statistic
​s​
, preferences and help
​'​
 modules inside misc to preserve modularity as pgAdmin is modular
​ ?​

Is there anything from a organization stance you discussed in the previous email that needs to be done to make this usable and consistent?
​No​


Thanks,

George & Sarah 





--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company





--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company





--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


Attachment

Re: [pgAdmin4]: Webpacking of static JS/CSS

From
Dave Page
Date:
That was a fun one to spot I'm sure!

Thanks, committed.

On Wed, Jul 19, 2017 at 11:21 AM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

Fix library path reference for `jquery.contextmenu'. This issue was only reproducible on Linux machine.
So, i setup pgAdmin4 on Ubuntu VM and tested the patch and it works.

Please find attached patch.

Thanks,
Surinder

On Tue, Jul 18, 2017 at 9:05 PM, Murtuza Zabuawala <murtuza.zabuawala@enterprisedb.com> wrote:
Great job Surinder, Load time ~2 sec on browser :)
 
Inline image 1


--
Regards,
Murtuza Zabuawala
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Tue, Jul 18, 2017 at 9:01 PM, Dave Page <dpage@pgadmin.org> wrote:
Thanks, applied.

On Tue, Jul 18, 2017 at 4:12 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

1. As Slickgrid has dependency of `jQuery-ui`, it was missed. now added.
2. Column sorting for collection nodes sometimes failing when clicked on different collection nodes.

Please find attached patch.

Thanks
Surinder 

On Tue, Jul 18, 2017 at 8:20 PM, Khushboo Vashi <khushboo.vashi@enterprisedb.com> wrote:


On Tue, Jul 18, 2017 at 7:46 PM, Dave Page <dpage@pgadmin.org> wrote:
Thanks - applied!

Awesome work - on an average of 3 tests on my Mac, load time reduced from 11.55s with v1.6 to 5.53s with GIT Head.
​Thanks to all​

Surinder, great work...
 
On Mon, Jul 17, 2017 at 5:57 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

Now all test cases are executing.
Please find updated patch.

Thanks
Surinder

On Mon, Jul 17, 2017 at 6:57 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
On Mon, Jul 17, 2017 at 4:52 PM, Dave Page <dpage@pgadmin.org> wrote:
Hi

No errors now, but do you know why JS tests are being skipped?
​No errors/warning in console even after settings `logLevel: config.LOG_DEBUG`. I am still debugging.

PhantomJS 2.1.1 (Mac OS X 0.0.0): Executed 4 of 216 (skipped 212) SUCCESS (0.085 secs / 0.046 secs)

Thanks!

On Mon, Jul 17, 2017 at 12:07 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
​Hi Dave,

I didn't removed the vendor modules when i ran regression test cases, so modules were being referenced from vendor dir and passed for me.
Now I have fixed path references and test cases are working.

Please find attached patch.

Thanks
Surinder​

On Mon, Jul 17, 2017 at 3:18 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

I'm currently working on first TODO: "Automatically handle static and template JS files"

As discussed with Ashesh, currently the paths to module id are written manually in webpack.config.js, instead the path defined in moudle's `def get_own_javascript()`  should be used.

So, we will be generating a paths.json file which will contain:

1. resolve > alias - path with reference to module id.(Static files)

2. externals - list of modules to be loaded dynamically on demand(Template files)

3. Shim module dependency

4. List of JS modules to be loaded in specified order.

Implementation:

To generate `paths.json` file, we will be using `Flask's test_client` to make an http request internally within the app context so we can call `current_app.javascripts` property and return the list of JS paths and write those into paths.json file and then use it in webpack.shim.js before the execution of `yarn run bundle` in `javascript_bundler.py`

For example:
@app.route('/get_script_paths')
def get_script_paths():
    from flask import current_app
    from pgadmin.utils.ajax import make_json_response

    return make_json_response(data=current_app.javascripts)

if config.DEBUG:
    with app.test_client() as client:
        import simplejson as json
        list_scripts = client.get('/get_script_paths')
        scripts = json.loads(list_scripts.data)

    javascriptBundler = JavascriptBundler()
    javascriptBundler.bundle(scripts['data'])

This also needs little change in module dependency we defined using 'When': 'node_name' in `def get_own_javascripts(...)` method
the module specified(name: module_name) is loaded when module given in `When` is expanded in node. Since we are using Webpack in which behaviour to load module is little different.

Now in webpack we are using `imports-loader` to load specific modules. So this is how it should work.

1. First load all modules which do not have dependency on any node like 'about', 'dashboard',  'server-group', 'server' etc.

2. Load module such as `Databases` node first before its child nodes are loaded.
Similarly load `Schemas` node before its child nodes are loaded as they are dependent on parent node.

Thanks,
Surinder
On Wed, Jul 5, 2017 at 8:22 PM, Sarah McAlear <smcalear@pivotal.io> wrote:
Hello,


​Things to discuss:

How to differentiate between a static and template JS
​​
.

What is the advantage of webpacking templated JS? It seems as though this creates a system in which the bundled dependencies have to refer back to the backend to load the templates. 
​Templated JS will not be part of generated bundle JS, they will load externally( an extra request will be made to server For example: translations.js)

If there is a performance win in packing templated JS then looking at it makes sense.  Otherwise it may make sense to put off until it is clear that the templated files should be dealt with by either de-templating them or bundling them where there is a clear reason.
​Template JS cannot be bundled, so i extract the <Jinja> code from template files and put into a separate file - ABC.js (also moved template files to static directory) and then load ABC.js dynamically as dependency of other modules.

However, we're wondering about possible performance penalties with templating larger files (as opposed to templating on-demand.) Since jinja templates can execute arbitrary python, this could get time expensive and further slow things like initial page-load.
Another concern is: what happens when a template gets out of date (e.g. if browser.js had previously filled in the content for 'panel_item.content' and had been cached, would it render a new version with the new values when needed? Or is it possible that we would get old content?) 
​That file will always gets new content when loaded dynamically, the content is not cached.​
 
Taks remaining:

​1. ​
Fix local variables which are declared without using var, have to check in each file
​ by​
 running eslint (For now, i will fix only errors which are giving error in browser).

​2. ​
Move non-template files from ’templates’ to ’static’ directory. List of
​ pending​
 modules is here: 
  • Tools (mostly all modules - 9 modules) 
  • Browser nodes - 3 modules(resource group, roles, tablespace)
  • ​About
    ​​
Also can we move 
​'​
dashboard, statistic
​s​
, preferences and help
​'​
 modules inside misc to preserve modularity as pgAdmin is modular
​ ?​

Is there anything from a organization stance you discussed in the previous email that needs to be done to make this usable and consistent?
​No​


Thanks,

George & Sarah 





--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company





--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company





--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company





--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Re: [pgAdmin4]: Webpacking of static JS/CSS

From
Surinder Kumar
Date:
On Wed, Jul 19, 2017 at 4:27 PM, Dave Page <dpage@pgadmin.org> wrote:
That was a fun one to spot I'm sure!
​Indeed, i had setup pgAdmin evn on Linux(as it works on Mac) and then i did `ls path/to/jquery.contextmenu.js`, it was spotted :)

Thanks, committed.

On Wed, Jul 19, 2017 at 11:21 AM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

Fix library path reference for `jquery.contextmenu'. This issue was only reproducible on Linux machine.
So, i setup pgAdmin4 on Ubuntu VM and tested the patch and it works.

Please find attached patch.

Thanks,
Surinder

On Tue, Jul 18, 2017 at 9:05 PM, Murtuza Zabuawala <murtuza.zabuawala@enterprisedb.com> wrote:
Great job Surinder, Load time ~2 sec on browser :)
 
Inline image 1


--
Regards,
Murtuza Zabuawala
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Tue, Jul 18, 2017 at 9:01 PM, Dave Page <dpage@pgadmin.org> wrote:
Thanks, applied.

On Tue, Jul 18, 2017 at 4:12 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

1. As Slickgrid has dependency of `jQuery-ui`, it was missed. now added.
2. Column sorting for collection nodes sometimes failing when clicked on different collection nodes.

Please find attached patch.

Thanks
Surinder 

On Tue, Jul 18, 2017 at 8:20 PM, Khushboo Vashi <khushboo.vashi@enterprisedb.com> wrote:


On Tue, Jul 18, 2017 at 7:46 PM, Dave Page <dpage@pgadmin.org> wrote:
Thanks - applied!

Awesome work - on an average of 3 tests on my Mac, load time reduced from 11.55s with v1.6 to 5.53s with GIT Head.
​Thanks to all​

Surinder, great work...
 
On Mon, Jul 17, 2017 at 5:57 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

Now all test cases are executing.
Please find updated patch.

Thanks
Surinder

On Mon, Jul 17, 2017 at 6:57 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
On Mon, Jul 17, 2017 at 4:52 PM, Dave Page <dpage@pgadmin.org> wrote:
Hi

No errors now, but do you know why JS tests are being skipped?
​No errors/warning in console even after settings `logLevel: config.LOG_DEBUG`. I am still debugging.

PhantomJS 2.1.1 (Mac OS X 0.0.0): Executed 4 of 216 (skipped 212) SUCCESS (0.085 secs / 0.046 secs)

Thanks!

On Mon, Jul 17, 2017 at 12:07 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
​Hi Dave,

I didn't removed the vendor modules when i ran regression test cases, so modules were being referenced from vendor dir and passed for me.
Now I have fixed path references and test cases are working.

Please find attached patch.

Thanks
Surinder​

On Mon, Jul 17, 2017 at 3:18 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

I'm currently working on first TODO: "Automatically handle static and template JS files"

As discussed with Ashesh, currently the paths to module id are written manually in webpack.config.js, instead the path defined in moudle's `def get_own_javascript()`  should be used.

So, we will be generating a paths.json file which will contain:

1. resolve > alias - path with reference to module id.(Static files)

2. externals - list of modules to be loaded dynamically on demand(Template files)

3. Shim module dependency

4. List of JS modules to be loaded in specified order.

Implementation:

To generate `paths.json` file, we will be using `Flask's test_client` to make an http request internally within the app context so we can call `current_app.javascripts` property and return the list of JS paths and write those into paths.json file and then use it in webpack.shim.js before the execution of `yarn run bundle` in `javascript_bundler.py`

For example:
@app.route('/get_script_paths')
def get_script_paths():
    from flask import current_app
    from pgadmin.utils.ajax import make_json_response

    return make_json_response(data=current_app.javascripts)

if config.DEBUG:
    with app.test_client() as client:
        import simplejson as json
        list_scripts = client.get('/get_script_paths')
        scripts = json.loads(list_scripts.data)

    javascriptBundler = JavascriptBundler()
    javascriptBundler.bundle(scripts['data'])

This also needs little change in module dependency we defined using 'When': 'node_name' in `def get_own_javascripts(...)` method
the module specified(name: module_name) is loaded when module given in `When` is expanded in node. Since we are using Webpack in which behaviour to load module is little different.

Now in webpack we are using `imports-loader` to load specific modules. So this is how it should work.

1. First load all modules which do not have dependency on any node like 'about', 'dashboard',  'server-group', 'server' etc.

2. Load module such as `Databases` node first before its child nodes are loaded.
Similarly load `Schemas` node before its child nodes are loaded as they are dependent on parent node.

Thanks,
Surinder
On Wed, Jul 5, 2017 at 8:22 PM, Sarah McAlear <smcalear@pivotal.io> wrote:
Hello,


​Things to discuss:

How to differentiate between a static and template JS
​​
.

What is the advantage of webpacking templated JS? It seems as though this creates a system in which the bundled dependencies have to refer back to the backend to load the templates. 
​Templated JS will not be part of generated bundle JS, they will load externally( an extra request will be made to server For example: translations.js)

If there is a performance win in packing templated JS then looking at it makes sense.  Otherwise it may make sense to put off until it is clear that the templated files should be dealt with by either de-templating them or bundling them where there is a clear reason.
​Template JS cannot be bundled, so i extract the <Jinja> code from template files and put into a separate file - ABC.js (also moved template files to static directory) and then load ABC.js dynamically as dependency of other modules.

However, we're wondering about possible performance penalties with templating larger files (as opposed to templating on-demand.) Since jinja templates can execute arbitrary python, this could get time expensive and further slow things like initial page-load.
Another concern is: what happens when a template gets out of date (e.g. if browser.js had previously filled in the content for 'panel_item.content' and had been cached, would it render a new version with the new values when needed? Or is it possible that we would get old content?) 
​That file will always gets new content when loaded dynamically, the content is not cached.​
 
Taks remaining:

​1. ​
Fix local variables which are declared without using var, have to check in each file
​ by​
 running eslint (For now, i will fix only errors which are giving error in browser).

​2. ​
Move non-template files from ’templates’ to ’static’ directory. List of
​ pending​
 modules is here: 
  • Tools (mostly all modules - 9 modules) 
  • Browser nodes - 3 modules(resource group, roles, tablespace)
  • ​About
    ​​
Also can we move 
​'​
dashboard, statistic
​s​
, preferences and help
​'​
 modules inside misc to preserve modularity as pgAdmin is modular
​ ?​

Is there anything from a organization stance you discussed in the previous email that needs to be done to make this usable and consistent?
​No​


Thanks,

George & Sarah 





--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company





--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company





--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company





--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Re: [pgAdmin4]: Webpacking of static JS/CSS

From
Surinder Kumar
Date:
Hi,

PFA patch for few code review comments given by Ashesh.

Thanks,
Surinder


On Wed, Jul 19, 2017 at 4:36 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
On Wed, Jul 19, 2017 at 4:27 PM, Dave Page <dpage@pgadmin.org> wrote:
That was a fun one to spot I'm sure!
​Indeed, i had setup pgAdmin evn on Linux(as it works on Mac) and then i did `ls path/to/jquery.contextmenu.js`, it was spotted :)

Thanks, committed.

On Wed, Jul 19, 2017 at 11:21 AM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

Fix library path reference for `jquery.contextmenu'. This issue was only reproducible on Linux machine.
So, i setup pgAdmin4 on Ubuntu VM and tested the patch and it works.

Please find attached patch.

Thanks,
Surinder

On Tue, Jul 18, 2017 at 9:05 PM, Murtuza Zabuawala <murtuza.zabuawala@enterprisedb.com> wrote:
Great job Surinder, Load time ~2 sec on browser :)
 
Inline image 1


--
Regards,
Murtuza Zabuawala
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Tue, Jul 18, 2017 at 9:01 PM, Dave Page <dpage@pgadmin.org> wrote:
Thanks, applied.

On Tue, Jul 18, 2017 at 4:12 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

1. As Slickgrid has dependency of `jQuery-ui`, it was missed. now added.
2. Column sorting for collection nodes sometimes failing when clicked on different collection nodes.

Please find attached patch.

Thanks
Surinder 

On Tue, Jul 18, 2017 at 8:20 PM, Khushboo Vashi <khushboo.vashi@enterprisedb.com> wrote:


On Tue, Jul 18, 2017 at 7:46 PM, Dave Page <dpage@pgadmin.org> wrote:
Thanks - applied!

Awesome work - on an average of 3 tests on my Mac, load time reduced from 11.55s with v1.6 to 5.53s with GIT Head.
​Thanks to all​

Surinder, great work...
 
On Mon, Jul 17, 2017 at 5:57 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

Now all test cases are executing.
Please find updated patch.

Thanks
Surinder

On Mon, Jul 17, 2017 at 6:57 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
On Mon, Jul 17, 2017 at 4:52 PM, Dave Page <dpage@pgadmin.org> wrote:
Hi

No errors now, but do you know why JS tests are being skipped?
​No errors/warning in console even after settings `logLevel: config.LOG_DEBUG`. I am still debugging.

PhantomJS 2.1.1 (Mac OS X 0.0.0): Executed 4 of 216 (skipped 212) SUCCESS (0.085 secs / 0.046 secs)

Thanks!

On Mon, Jul 17, 2017 at 12:07 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
​Hi Dave,

I didn't removed the vendor modules when i ran regression test cases, so modules were being referenced from vendor dir and passed for me.
Now I have fixed path references and test cases are working.

Please find attached patch.

Thanks
Surinder​

On Mon, Jul 17, 2017 at 3:18 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

I'm currently working on first TODO: "Automatically handle static and template JS files"

As discussed with Ashesh, currently the paths to module id are written manually in webpack.config.js, instead the path defined in moudle's `def get_own_javascript()`  should be used.

So, we will be generating a paths.json file which will contain:

1. resolve > alias - path with reference to module id.(Static files)

2. externals - list of modules to be loaded dynamically on demand(Template files)

3. Shim module dependency

4. List of JS modules to be loaded in specified order.

Implementation:

To generate `paths.json` file, we will be using `Flask's test_client` to make an http request internally within the app context so we can call `current_app.javascripts` property and return the list of JS paths and write those into paths.json file and then use it in webpack.shim.js before the execution of `yarn run bundle` in `javascript_bundler.py`

For example:
@app.route('/get_script_paths')
def get_script_paths():
    from flask import current_app
    from pgadmin.utils.ajax import make_json_response

    return make_json_response(data=current_app.javascripts)

if config.DEBUG:
    with app.test_client() as client:
        import simplejson as json
        list_scripts = client.get('/get_script_paths')
        scripts = json.loads(list_scripts.data)

    javascriptBundler = JavascriptBundler()
    javascriptBundler.bundle(scripts['data'])

This also needs little change in module dependency we defined using 'When': 'node_name' in `def get_own_javascripts(...)` method
the module specified(name: module_name) is loaded when module given in `When` is expanded in node. Since we are using Webpack in which behaviour to load module is little different.

Now in webpack we are using `imports-loader` to load specific modules. So this is how it should work.

1. First load all modules which do not have dependency on any node like 'about', 'dashboard',  'server-group', 'server' etc.

2. Load module such as `Databases` node first before its child nodes are loaded.
Similarly load `Schemas` node before its child nodes are loaded as they are dependent on parent node.

Thanks,
Surinder
On Wed, Jul 5, 2017 at 8:22 PM, Sarah McAlear <smcalear@pivotal.io> wrote:
Hello,


​Things to discuss:

How to differentiate between a static and template JS
​​
.

What is the advantage of webpacking templated JS? It seems as though this creates a system in which the bundled dependencies have to refer back to the backend to load the templates. 
​Templated JS will not be part of generated bundle JS, they will load externally( an extra request will be made to server For example: translations.js)

If there is a performance win in packing templated JS then looking at it makes sense.  Otherwise it may make sense to put off until it is clear that the templated files should be dealt with by either de-templating them or bundling them where there is a clear reason.
​Template JS cannot be bundled, so i extract the <Jinja> code from template files and put into a separate file - ABC.js (also moved template files to static directory) and then load ABC.js dynamically as dependency of other modules.

However, we're wondering about possible performance penalties with templating larger files (as opposed to templating on-demand.) Since jinja templates can execute arbitrary python, this could get time expensive and further slow things like initial page-load.
Another concern is: what happens when a template gets out of date (e.g. if browser.js had previously filled in the content for 'panel_item.content' and had been cached, would it render a new version with the new values when needed? Or is it possible that we would get old content?) 
​That file will always gets new content when loaded dynamically, the content is not cached.​
 
Taks remaining:

​1. ​
Fix local variables which are declared without using var, have to check in each file
​ by​
 running eslint (For now, i will fix only errors which are giving error in browser).

​2. ​
Move non-template files from ’templates’ to ’static’ directory. List of
​ pending​
 modules is here: 
  • Tools (mostly all modules - 9 modules) 
  • Browser nodes - 3 modules(resource group, roles, tablespace)
  • ​About
    ​​
Also can we move 
​'​
dashboard, statistic
​s​
, preferences and help
​'​
 modules inside misc to preserve modularity as pgAdmin is modular
​ ?​

Is there anything from a organization stance you discussed in the previous email that needs to be done to make this usable and consistent?
​No​


Thanks,

George & Sarah 





--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company





--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company





--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company





--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


Attachment

Re: [pgAdmin4]: Webpacking of static JS/CSS

From
Dave Page
Date:
Thanks, applied.

On Wed, Jul 19, 2017 at 2:40 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi,

PFA patch for few code review comments given by Ashesh.

Thanks,
Surinder


On Wed, Jul 19, 2017 at 4:36 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
On Wed, Jul 19, 2017 at 4:27 PM, Dave Page <dpage@pgadmin.org> wrote:
That was a fun one to spot I'm sure!
​Indeed, i had setup pgAdmin evn on Linux(as it works on Mac) and then i did `ls path/to/jquery.contextmenu.js`, it was spotted :)

Thanks, committed.

On Wed, Jul 19, 2017 at 11:21 AM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

Fix library path reference for `jquery.contextmenu'. This issue was only reproducible on Linux machine.
So, i setup pgAdmin4 on Ubuntu VM and tested the patch and it works.

Please find attached patch.

Thanks,
Surinder

On Tue, Jul 18, 2017 at 9:05 PM, Murtuza Zabuawala <murtuza.zabuawala@enterprisedb.com> wrote:
Great job Surinder, Load time ~2 sec on browser :)
 
Inline image 1


--
Regards,
Murtuza Zabuawala
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Tue, Jul 18, 2017 at 9:01 PM, Dave Page <dpage@pgadmin.org> wrote:
Thanks, applied.

On Tue, Jul 18, 2017 at 4:12 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

1. As Slickgrid has dependency of `jQuery-ui`, it was missed. now added.
2. Column sorting for collection nodes sometimes failing when clicked on different collection nodes.

Please find attached patch.

Thanks
Surinder 

On Tue, Jul 18, 2017 at 8:20 PM, Khushboo Vashi <khushboo.vashi@enterprisedb.com> wrote:


On Tue, Jul 18, 2017 at 7:46 PM, Dave Page <dpage@pgadmin.org> wrote:
Thanks - applied!

Awesome work - on an average of 3 tests on my Mac, load time reduced from 11.55s with v1.6 to 5.53s with GIT Head.
​Thanks to all​

Surinder, great work...
 
On Mon, Jul 17, 2017 at 5:57 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

Now all test cases are executing.
Please find updated patch.

Thanks
Surinder

On Mon, Jul 17, 2017 at 6:57 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
On Mon, Jul 17, 2017 at 4:52 PM, Dave Page <dpage@pgadmin.org> wrote:
Hi

No errors now, but do you know why JS tests are being skipped?
​No errors/warning in console even after settings `logLevel: config.LOG_DEBUG`. I am still debugging.

PhantomJS 2.1.1 (Mac OS X 0.0.0): Executed 4 of 216 (skipped 212) SUCCESS (0.085 secs / 0.046 secs)

Thanks!

On Mon, Jul 17, 2017 at 12:07 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
​Hi Dave,

I didn't removed the vendor modules when i ran regression test cases, so modules were being referenced from vendor dir and passed for me.
Now I have fixed path references and test cases are working.

Please find attached patch.

Thanks
Surinder​

On Mon, Jul 17, 2017 at 3:18 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

I'm currently working on first TODO: "Automatically handle static and template JS files"

As discussed with Ashesh, currently the paths to module id are written manually in webpack.config.js, instead the path defined in moudle's `def get_own_javascript()`  should be used.

So, we will be generating a paths.json file which will contain:

1. resolve > alias - path with reference to module id.(Static files)

2. externals - list of modules to be loaded dynamically on demand(Template files)

3. Shim module dependency

4. List of JS modules to be loaded in specified order.

Implementation:

To generate `paths.json` file, we will be using `Flask's test_client` to make an http request internally within the app context so we can call `current_app.javascripts` property and return the list of JS paths and write those into paths.json file and then use it in webpack.shim.js before the execution of `yarn run bundle` in `javascript_bundler.py`

For example:
@app.route('/get_script_paths')
def get_script_paths():
    from flask import current_app
    from pgadmin.utils.ajax import make_json_response

    return make_json_response(data=current_app.javascripts)

if config.DEBUG:
    with app.test_client() as client:
        import simplejson as json
        list_scripts = client.get('/get_script_paths')
        scripts = json.loads(list_scripts.data)

    javascriptBundler = JavascriptBundler()
    javascriptBundler.bundle(scripts['data'])

This also needs little change in module dependency we defined using 'When': 'node_name' in `def get_own_javascripts(...)` method
the module specified(name: module_name) is loaded when module given in `When` is expanded in node. Since we are using Webpack in which behaviour to load module is little different.

Now in webpack we are using `imports-loader` to load specific modules. So this is how it should work.

1. First load all modules which do not have dependency on any node like 'about', 'dashboard',  'server-group', 'server' etc.

2. Load module such as `Databases` node first before its child nodes are loaded.
Similarly load `Schemas` node before its child nodes are loaded as they are dependent on parent node.

Thanks,
Surinder
On Wed, Jul 5, 2017 at 8:22 PM, Sarah McAlear <smcalear@pivotal.io> wrote:
Hello,


​Things to discuss:

How to differentiate between a static and template JS
​​
.

What is the advantage of webpacking templated JS? It seems as though this creates a system in which the bundled dependencies have to refer back to the backend to load the templates. 
​Templated JS will not be part of generated bundle JS, they will load externally( an extra request will be made to server For example: translations.js)

If there is a performance win in packing templated JS then looking at it makes sense.  Otherwise it may make sense to put off until it is clear that the templated files should be dealt with by either de-templating them or bundling them where there is a clear reason.
​Template JS cannot be bundled, so i extract the <Jinja> code from template files and put into a separate file - ABC.js (also moved template files to static directory) and then load ABC.js dynamically as dependency of other modules.

However, we're wondering about possible performance penalties with templating larger files (as opposed to templating on-demand.) Since jinja templates can execute arbitrary python, this could get time expensive and further slow things like initial page-load.
Another concern is: what happens when a template gets out of date (e.g. if browser.js had previously filled in the content for 'panel_item.content' and had been cached, would it render a new version with the new values when needed? Or is it possible that we would get old content?) 
​That file will always gets new content when loaded dynamically, the content is not cached.​
 
Taks remaining:

​1. ​
Fix local variables which are declared without using var, have to check in each file
​ by​
 running eslint (For now, i will fix only errors which are giving error in browser).

​2. ​
Move non-template files from ’templates’ to ’static’ directory. List of
​ pending​
 modules is here: 
  • Tools (mostly all modules - 9 modules) 
  • Browser nodes - 3 modules(resource group, roles, tablespace)
  • ​About
    ​​
Also can we move 
​'​
dashboard, statistic
​s​
, preferences and help
​'​
 modules inside misc to preserve modularity as pgAdmin is modular
​ ?​

Is there anything from a organization stance you discussed in the previous email that needs to be done to make this usable and consistent?
​No​


Thanks,

George & Sarah 





--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company





--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company





--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company





--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company





--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Re: [pgAdmin4]: Webpacking of static JS/CSS

From
Surinder Kumar
Date:
Hi

The loading icon image is used in FileManager and it is still referenced from vendor `aciTree`(which is removed) so it gives 404 not found when FileManager is opened from Query tool/Backup utility.

Please find attached patch and review.

Thanks,
Surinder Kumar
Attachment

Re: [pgAdmin4]: Webpacking of static JS/CSS

From
Surinder Kumar
Date:
Hi

Changes:

Add proper comments in webpack configuration files.
Rename lib_css to style.css

Please find attached patch and review

Thanks,
Surinder

On Thu, Jul 20, 2017 at 12:37 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

The loading icon image is used in FileManager and it is still referenced from vendor `aciTree`(which is removed) so it gives 404 not found when FileManager is opened from Query tool/Backup utility.

Please find attached patch and review.

Thanks,
Surinder Kumar

Attachment

Re: [pgAdmin4]: Webpacking of static JS/CSS

From
Surinder Kumar
Date:
Hi Dave,

Can you please review this patch please? I guess it was missed as i send 2 patches back to back in same email chain.
Please find inline detailed description of issue.

On Thu, Jul 20, 2017 at 12:37 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

The loading icon image is used in FileManager and it is still referenced from vendor `aciTree`(which is removed) so it gives 404 not found when FileManager is opened from Query tool/Backup utility.

Please find attached patch and review.

Thanks,
Surinder Kumar

Attachment

Re: [pgAdmin4]: Webpacking of static JS/CSS

From
Dave Page
Date:
Thanks, applied.

On Thu, Jul 20, 2017 at 1:35 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi Dave,

Can you please review this patch please? I guess it was missed as i send 2 patches back to back in same email chain.
Please find inline detailed description of issue.

On Thu, Jul 20, 2017 at 12:37 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

The loading icon image is used in FileManager and it is still referenced from vendor `aciTree`(which is removed) so it gives 404 not found when FileManager is opened from Query tool/Backup utility.

Please find attached patch and review.

Thanks,
Surinder Kumar




--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Re: [pgAdmin4]: Webpacking of static JS/CSS

From
Surinder Kumar
Date:
Hi,

Following are the changes in patches attached:

1) Move Javascripts from Templates directory to Static for following modules:
 - About
 - Browser nodes
 - Dashboard
 - FileManager
 - Vendor/snap.svg
 - Preferences
 - Settings
 - Backup
 - Datagrid
 - Debugger
 - Sqleditor
 - Grant Wizard
 - Import & Export
 - Maintenance
 - Restore and
 - User Management

2) Generate module JS path references dynamically/ Remove manually written paths.
Added a new file 'module_paths.json' which will update with absolute paths to module JS defined in every modules > __init__.py > 'def get_own_javascripts'.

When Flask service is started, it accesses the all module javascripts using `current_app.javascripts` which is called inside an route.
This call to route is made using Flask's `test_client` and thus those paths are written to `module_paths.json`(in javascript_bundler.py) which is used by webpack.shim.js file.

Three patches:
1. Move JS files to static directory
2. Deleted template JS
3. Generate JS paths before app starts

These patches needs to be committed separately as deleted files includes in the commit that makes difficult to look for changes through `git log <commit-has>`

Please review.

Thanks,
Surinder



On Thu, Jul 20, 2017 at 6:08 PM, Dave Page <dpage@pgadmin.org> wrote:
Thanks, applied.

On Thu, Jul 20, 2017 at 1:35 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi Dave,

Can you please review this patch please? I guess it was missed as i send 2 patches back to back in same email chain.
Please find inline detailed description of issue.

On Thu, Jul 20, 2017 at 12:37 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

The loading icon image is used in FileManager and it is still referenced from vendor `aciTree`(which is removed) so it gives 404 not found when FileManager is opened from Query tool/Backup utility.

Please find attached patch and review.

Thanks,
Surinder Kumar




--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Attachment

Re: [pgAdmin4]: Webpacking of static JS/CSS

From
Surinder Kumar
Date:
Hi,

Please find rebased patches and review.

On Sat, Jul 22, 2017 at 1:03 AM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi,

Following are the changes in patches attached:

1) Move Javascripts from Templates directory to Static for following modules:
 - About
 - Browser nodes
 - Dashboard
 - FileManager
 - Vendor/snap.svg
 - Preferences
 - Settings
 - Backup
 - Datagrid
 - Debugger
 - Sqleditor
 - Grant Wizard
 - Import & Export
 - Maintenance
 - Restore and
 - User Management

2) Generate module JS path references dynamically/ Remove manually written paths.
Added a new file 'module_paths.json' which will update with absolute paths to module JS defined in every modules > __init__.py > 'def get_own_javascripts'.

When Flask service is started, it accesses the all module javascripts using `current_app.javascripts` which is called inside an route.
This call to route is made using Flask's `test_client` and thus those paths are written to `module_paths.json`(in javascript_bundler.py) which is used by webpack.shim.js file.

Three patches:
1. Move JS files to static directory
2. Deleted template JS
3. Generate JS paths before app starts

These patches needs to be committed separately as deleted files includes in the commit that makes difficult to look for changes through `git log <commit-has>`

Please review.

Thanks,
Surinder



On Thu, Jul 20, 2017 at 6:08 PM, Dave Page <dpage@pgadmin.org> wrote:
Thanks, applied.

On Thu, Jul 20, 2017 at 1:35 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi Dave,

Can you please review this patch please? I guess it was missed as i send 2 patches back to back in same email chain.
Please find inline detailed description of issue.

On Thu, Jul 20, 2017 at 12:37 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

The loading icon image is used in FileManager and it is still referenced from vendor `aciTree`(which is removed) so it gives 404 not found when FileManager is opened from Query tool/Backup utility.

Please find attached patch and review.

Thanks,
Surinder Kumar




--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


Attachment

Re: [pgAdmin4]: Webpacking of static JS/CSS

From
Ashesh Vashi
Date:
Hi Surinder,

'delete_template_files.patch' is applied any more.
Please send the rebased patches.

I have also noticed in the 'move_js_files_to_static_directory.patch', couple of modules have two definitions for 'module_use_template_javascript' method.


--

Thanks & Regards,

Ashesh Vashi
EnterpriseDB INDIA: Enterprise PostgreSQL Company


http://www.linkedin.com/in/asheshvashi


On Tue, Jul 25, 2017 at 3:56 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi,

Please find rebased patches and review.

On Sat, Jul 22, 2017 at 1:03 AM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi,

Following are the changes in patches attached:

1) Move Javascripts from Templates directory to Static for following modules:
 - About
 - Browser nodes
 - Dashboard
 - FileManager
 - Vendor/snap.svg
 - Preferences
 - Settings
 - Backup
 - Datagrid
 - Debugger
 - Sqleditor
 - Grant Wizard
 - Import & Export
 - Maintenance
 - Restore and
 - User Management

2) Generate module JS path references dynamically/ Remove manually written paths.
Added a new file 'module_paths.json' which will update with absolute paths to module JS defined in every modules > __init__.py > 'def get_own_javascripts'.

When Flask service is started, it accesses the all module javascripts using `current_app.javascripts` which is called inside an route.
This call to route is made using Flask's `test_client` and thus those paths are written to `module_paths.json`(in javascript_bundler.py) which is used by webpack.shim.js file.

Three patches:
1. Move JS files to static directory
2. Deleted template JS
3. Generate JS paths before app starts

These patches needs to be committed separately as deleted files includes in the commit that makes difficult to look for changes through `git log <commit-has>`

Please review.

Thanks,
Surinder



On Thu, Jul 20, 2017 at 6:08 PM, Dave Page <dpage@pgadmin.org> wrote:
Thanks, applied.

On Thu, Jul 20, 2017 at 1:35 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi Dave,

Can you please review this patch please? I guess it was missed as i send 2 patches back to back in same email chain.
Please find inline detailed description of issue.

On Thu, Jul 20, 2017 at 12:37 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

The loading icon image is used in FileManager and it is still referenced from vendor `aciTree`(which is removed) so it gives 404 not found when FileManager is opened from Query tool/Backup utility.

Please find attached patch and review.

Thanks,
Surinder Kumar




--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company



Re: [pgAdmin4]: Webpacking of static JS/CSS

From
Ashesh Vashi
Date:


On Thu, Jul 27, 2017 at 10:22 AM, Ashesh Vashi <ashesh.vashi@enterprisedb.com> wrote:
Hi Surinder,

'delete_template_files.patch' is applied any more.
Please send the rebased patches.

I have also noticed in the 'move_js_files_to_static_directory.patch', couple of modules have two definitions for 'module_use_template_javascript' method.
Also - 'module_paths.json' is auto generated, please don't include that in the patch.

-- Thanks, Ashesh 


--

Thanks & Regards,

Ashesh Vashi
EnterpriseDB INDIA: Enterprise PostgreSQL Company


http://www.linkedin.com/in/asheshvashi


On Tue, Jul 25, 2017 at 3:56 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi,

Please find rebased patches and review.

On Sat, Jul 22, 2017 at 1:03 AM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi,

Following are the changes in patches attached:

1) Move Javascripts from Templates directory to Static for following modules:
 - About
 - Browser nodes
 - Dashboard
 - FileManager
 - Vendor/snap.svg
 - Preferences
 - Settings
 - Backup
 - Datagrid
 - Debugger
 - Sqleditor
 - Grant Wizard
 - Import & Export
 - Maintenance
 - Restore and
 - User Management

2) Generate module JS path references dynamically/ Remove manually written paths.
Added a new file 'module_paths.json' which will update with absolute paths to module JS defined in every modules > __init__.py > 'def get_own_javascripts'.

When Flask service is started, it accesses the all module javascripts using `current_app.javascripts` which is called inside an route.
This call to route is made using Flask's `test_client` and thus those paths are written to `module_paths.json`(in javascript_bundler.py) which is used by webpack.shim.js file.

Three patches:
1. Move JS files to static directory
2. Deleted template JS
3. Generate JS paths before app starts

These patches needs to be committed separately as deleted files includes in the commit that makes difficult to look for changes through `git log <commit-has>`

Please review.

Thanks,
Surinder



On Thu, Jul 20, 2017 at 6:08 PM, Dave Page <dpage@pgadmin.org> wrote:
Thanks, applied.

On Thu, Jul 20, 2017 at 1:35 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi Dave,

Can you please review this patch please? I guess it was missed as i send 2 patches back to back in same email chain.
Please find inline detailed description of issue.

On Thu, Jul 20, 2017 at 12:37 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

The loading icon image is used in FileManager and it is still referenced from vendor `aciTree`(which is removed) so it gives 404 not found when FileManager is opened from Query tool/Backup utility.

Please find attached patch and review.

Thanks,
Surinder Kumar




--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company




Re: [pgAdmin4]: Webpacking of static JS/CSS

From
Surinder Kumar
Date:
Ok. I will send rebased patch.

On Thu, Jul 27, 2017 at 10:32 AM, Ashesh Vashi <ashesh.vashi@enterprisedb.com> wrote:


On Thu, Jul 27, 2017 at 10:22 AM, Ashesh Vashi <ashesh.vashi@enterprisedb.com> wrote:
Hi Surinder,

'delete_template_files.patch' is applied any more.
Please send the rebased patches.

I have also noticed in the 'move_js_files_to_static_directory.patch', couple of modules have two definitions for 'module_use_template_javascript' method.
Also - 'module_paths.json' is auto generated, please don't include that in the patch.

-- Thanks, Ashesh 


--

Thanks & Regards,

Ashesh Vashi
EnterpriseDB INDIA: Enterprise PostgreSQL Company


http://www.linkedin.com/in/asheshvashi


On Tue, Jul 25, 2017 at 3:56 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi,

Please find rebased patches and review.

On Sat, Jul 22, 2017 at 1:03 AM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi,

Following are the changes in patches attached:

1) Move Javascripts from Templates directory to Static for following modules:
 - About
 - Browser nodes
 - Dashboard
 - FileManager
 - Vendor/snap.svg
 - Preferences
 - Settings
 - Backup
 - Datagrid
 - Debugger
 - Sqleditor
 - Grant Wizard
 - Import & Export
 - Maintenance
 - Restore and
 - User Management

2) Generate module JS path references dynamically/ Remove manually written paths.
Added a new file 'module_paths.json' which will update with absolute paths to module JS defined in every modules > __init__.py > 'def get_own_javascripts'.

When Flask service is started, it accesses the all module javascripts using `current_app.javascripts` which is called inside an route.
This call to route is made using Flask's `test_client` and thus those paths are written to `module_paths.json`(in javascript_bundler.py) which is used by webpack.shim.js file.

Three patches:
1. Move JS files to static directory
2. Deleted template JS
3. Generate JS paths before app starts

These patches needs to be committed separately as deleted files includes in the commit that makes difficult to look for changes through `git log <commit-has>`

Please review.

Thanks,
Surinder



On Thu, Jul 20, 2017 at 6:08 PM, Dave Page <dpage@pgadmin.org> wrote:
Thanks, applied.

On Thu, Jul 20, 2017 at 1:35 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi Dave,

Can you please review this patch please? I guess it was missed as i send 2 patches back to back in same email chain.
Please find inline detailed description of issue.

On Thu, Jul 20, 2017 at 12:37 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

The loading icon image is used in FileManager and it is still referenced from vendor `aciTree`(which is removed) so it gives 404 not found when FileManager is opened from Query tool/Backup utility.

Please find attached patch and review.

Thanks,
Surinder Kumar




--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company





Re: [pgAdmin4]: Webpacking of static JS/CSS

From
Surinder Kumar
Date:
Hi Ashesh,

Changes in patch:

1. I have removed duplicate function 'module_use_template_javascript' from modules.
2. This patches only contains changes for moving template/js to static/js. I have excluded changes for making module paths dynamic as we will be using different approach to handle that.
3. Also, changed path references for JS modules to point to static/ dir instead of template/js in webpack.shim.js and webpack.config.js .

Please find attached patches

Thanks,
Surinder


On Thu, Jul 27, 2017 at 10:33 AM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Ok. I will send rebased patch.

On Thu, Jul 27, 2017 at 10:32 AM, Ashesh Vashi <ashesh.vashi@enterprisedb.com> wrote:


On Thu, Jul 27, 2017 at 10:22 AM, Ashesh Vashi <ashesh.vashi@enterprisedb.com> wrote:
Hi Surinder,

'delete_template_files.patch' is applied any more.
Please send the rebased patches.

I have also noticed in the 'move_js_files_to_static_directory.patch', couple of modules have two definitions for 'module_use_template_javascript' method.
Also - 'module_paths.json' is auto generated, please don't include that in the patch.

-- Thanks, Ashesh 


--

Thanks & Regards,

Ashesh Vashi
EnterpriseDB INDIA: Enterprise PostgreSQL Company


http://www.linkedin.com/in/asheshvashi


On Tue, Jul 25, 2017 at 3:56 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi,

Please find rebased patches and review.

On Sat, Jul 22, 2017 at 1:03 AM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi,

Following are the changes in patches attached:

1) Move Javascripts from Templates directory to Static for following modules:
 - About
 - Browser nodes
 - Dashboard
 - FileManager
 - Vendor/snap.svg
 - Preferences
 - Settings
 - Backup
 - Datagrid
 - Debugger
 - Sqleditor
 - Grant Wizard
 - Import & Export
 - Maintenance
 - Restore and
 - User Management

2) Generate module JS path references dynamically/ Remove manually written paths.
Added a new file 'module_paths.json' which will update with absolute paths to module JS defined in every modules > __init__.py > 'def get_own_javascripts'.

When Flask service is started, it accesses the all module javascripts using `current_app.javascripts` which is called inside an route.
This call to route is made using Flask's `test_client` and thus those paths are written to `module_paths.json`(in javascript_bundler.py) which is used by webpack.shim.js file.

Three patches:
1. Move JS files to static directory
2. Deleted template JS
3. Generate JS paths before app starts

These patches needs to be committed separately as deleted files includes in the commit that makes difficult to look for changes through `git log <commit-has>`

Please review.

Thanks,
Surinder



On Thu, Jul 20, 2017 at 6:08 PM, Dave Page <dpage@pgadmin.org> wrote:
Thanks, applied.

On Thu, Jul 20, 2017 at 1:35 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi Dave,

Can you please review this patch please? I guess it was missed as i send 2 patches back to back in same email chain.
Please find inline detailed description of issue.

On Thu, Jul 20, 2017 at 12:37 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

The loading icon image is used in FileManager and it is still referenced from vendor `aciTree`(which is removed) so it gives 404 not found when FileManager is opened from Query tool/Backup utility.

Please find attached patch and review.

Thanks,
Surinder Kumar




--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company






Attachment

Re: [pgAdmin4]: Webpacking of static JS/CSS

From
Surinder Kumar
Date:
Hi Ashesh,

Please find attached patch 'move_js_files_static_dir' with --binary option.

Thanks,
Surinder

On Thu, Jul 27, 2017 at 12:09 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi Ashesh,

Changes in patch:

1. I have removed duplicate function 'module_use_template_javascript' from modules.
2. This patches only contains changes for moving template/js to static/js. I have excluded changes for making module paths dynamic as we will be using different approach to handle that.
3. Also, changed path references for JS modules to point to static/ dir instead of template/js in webpack.shim.js and webpack.config.js .

Please find attached patches

Thanks,
Surinder


On Thu, Jul 27, 2017 at 10:33 AM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Ok. I will send rebased patch.

On Thu, Jul 27, 2017 at 10:32 AM, Ashesh Vashi <ashesh.vashi@enterprisedb.com> wrote:


On Thu, Jul 27, 2017 at 10:22 AM, Ashesh Vashi <ashesh.vashi@enterprisedb.com> wrote:
Hi Surinder,

'delete_template_files.patch' is applied any more.
Please send the rebased patches.

I have also noticed in the 'move_js_files_to_static_directory.patch', couple of modules have two definitions for 'module_use_template_javascript' method.
Also - 'module_paths.json' is auto generated, please don't include that in the patch.

-- Thanks, Ashesh 


--

Thanks & Regards,

Ashesh Vashi
EnterpriseDB INDIA: Enterprise PostgreSQL Company


http://www.linkedin.com/in/asheshvashi


On Tue, Jul 25, 2017 at 3:56 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi,

Please find rebased patches and review.

On Sat, Jul 22, 2017 at 1:03 AM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi,

Following are the changes in patches attached:

1) Move Javascripts from Templates directory to Static for following modules:
 - About
 - Browser nodes
 - Dashboard
 - FileManager
 - Vendor/snap.svg
 - Preferences
 - Settings
 - Backup
 - Datagrid
 - Debugger
 - Sqleditor
 - Grant Wizard
 - Import & Export
 - Maintenance
 - Restore and
 - User Management

2) Generate module JS path references dynamically/ Remove manually written paths.
Added a new file 'module_paths.json' which will update with absolute paths to module JS defined in every modules > __init__.py > 'def get_own_javascripts'.

When Flask service is started, it accesses the all module javascripts using `current_app.javascripts` which is called inside an route.
This call to route is made using Flask's `test_client` and thus those paths are written to `module_paths.json`(in javascript_bundler.py) which is used by webpack.shim.js file.

Three patches:
1. Move JS files to static directory
2. Deleted template JS
3. Generate JS paths before app starts

These patches needs to be committed separately as deleted files includes in the commit that makes difficult to look for changes through `git log <commit-has>`

Please review.

Thanks,
Surinder



On Thu, Jul 20, 2017 at 6:08 PM, Dave Page <dpage@pgadmin.org> wrote:
Thanks, applied.

On Thu, Jul 20, 2017 at 1:35 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi Dave,

Can you please review this patch please? I guess it was missed as i send 2 patches back to back in same email chain.
Please find inline detailed description of issue.

On Thu, Jul 20, 2017 at 12:37 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

The loading icon image is used in FileManager and it is still referenced from vendor `aciTree`(which is removed) so it gives 404 not found when FileManager is opened from Query tool/Backup utility.

Please find attached patch and review.

Thanks,
Surinder Kumar




--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company







Attachment

Re: [pgAdmin4]: Webpacking of static JS/CSS

From
Ashesh Vashi
Date:
On Thu, Jul 27, 2017 at 12:57 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi Ashesh,

Please find attached patch 'move_js_files_static_dir' with --binary option.
As discussed, this patch should include changes related to moving the file to static directory only, and not related to the dynamic path generation.

-- Thanks, Ashesh 

Thanks,
Surinder

On Thu, Jul 27, 2017 at 12:09 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi Ashesh,

Changes in patch:

1. I have removed duplicate function 'module_use_template_javascript' from modules.
2. This patches only contains changes for moving template/js to static/js. I have excluded changes for making module paths dynamic as we will be using different approach to handle that.
3. Also, changed path references for JS modules to point to static/ dir instead of template/js in webpack.shim.js and webpack.config.js .

Please find attached patches

Thanks,
Surinder


On Thu, Jul 27, 2017 at 10:33 AM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Ok. I will send rebased patch.

On Thu, Jul 27, 2017 at 10:32 AM, Ashesh Vashi <ashesh.vashi@enterprisedb.com> wrote:


On Thu, Jul 27, 2017 at 10:22 AM, Ashesh Vashi <ashesh.vashi@enterprisedb.com> wrote:
Hi Surinder,

'delete_template_files.patch' is applied any more.
Please send the rebased patches.

I have also noticed in the 'move_js_files_to_static_directory.patch', couple of modules have two definitions for 'module_use_template_javascript' method.
Also - 'module_paths.json' is auto generated, please don't include that in the patch.

-- Thanks, Ashesh 


--

Thanks & Regards,

Ashesh Vashi
EnterpriseDB INDIA: Enterprise PostgreSQL Company


http://www.linkedin.com/in/asheshvashi


On Tue, Jul 25, 2017 at 3:56 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi,

Please find rebased patches and review.

On Sat, Jul 22, 2017 at 1:03 AM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi,

Following are the changes in patches attached:

1) Move Javascripts from Templates directory to Static for following modules:
 - About
 - Browser nodes
 - Dashboard
 - FileManager
 - Vendor/snap.svg
 - Preferences
 - Settings
 - Backup
 - Datagrid
 - Debugger
 - Sqleditor
 - Grant Wizard
 - Import & Export
 - Maintenance
 - Restore and
 - User Management

2) Generate module JS path references dynamically/ Remove manually written paths.
Added a new file 'module_paths.json' which will update with absolute paths to module JS defined in every modules > __init__.py > 'def get_own_javascripts'.

When Flask service is started, it accesses the all module javascripts using `current_app.javascripts` which is called inside an route.
This call to route is made using Flask's `test_client` and thus those paths are written to `module_paths.json`(in javascript_bundler.py) which is used by webpack.shim.js file.

Three patches:
1. Move JS files to static directory
2. Deleted template JS
3. Generate JS paths before app starts

These patches needs to be committed separately as deleted files includes in the commit that makes difficult to look for changes through `git log <commit-has>`

Please review.

Thanks,
Surinder



On Thu, Jul 20, 2017 at 6:08 PM, Dave Page <dpage@pgadmin.org> wrote:
Thanks, applied.

On Thu, Jul 20, 2017 at 1:35 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi Dave,

Can you please review this patch please? I guess it was missed as i send 2 patches back to back in same email chain.
Please find inline detailed description of issue.

On Thu, Jul 20, 2017 at 12:37 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

The loading icon image is used in FileManager and it is still referenced from vendor `aciTree`(which is removed) so it gives 404 not found when FileManager is opened from Query tool/Backup utility.

Please find attached patch and review.

Thanks,
Surinder Kumar




--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company








Re: [pgAdmin4]: Webpacking of static JS/CSS

From
Surinder Kumar
Date:
Hi Ashesh,

I have removed changes which are not relevant to the moving file to static directory.

Please find updated patch.

Thanks,
Surinder

On Thu, Jul 27, 2017 at 1:13 PM, Ashesh Vashi <ashesh.vashi@enterprisedb.com> wrote:
On Thu, Jul 27, 2017 at 12:57 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi Ashesh,

Please find attached patch 'move_js_files_static_dir' with --binary option.
As discussed, this patch should include changes related to moving the file to static directory only, and not related to the dynamic path generation.

-- Thanks, Ashesh 

Thanks,
Surinder

On Thu, Jul 27, 2017 at 12:09 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi Ashesh,

Changes in patch:

1. I have removed duplicate function 'module_use_template_javascript' from modules.
2. This patches only contains changes for moving template/js to static/js. I have excluded changes for making module paths dynamic as we will be using different approach to handle that.
3. Also, changed path references for JS modules to point to static/ dir instead of template/js in webpack.shim.js and webpack.config.js .

Please find attached patches

Thanks,
Surinder


On Thu, Jul 27, 2017 at 10:33 AM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Ok. I will send rebased patch.

On Thu, Jul 27, 2017 at 10:32 AM, Ashesh Vashi <ashesh.vashi@enterprisedb.com> wrote:


On Thu, Jul 27, 2017 at 10:22 AM, Ashesh Vashi <ashesh.vashi@enterprisedb.com> wrote:
Hi Surinder,

'delete_template_files.patch' is applied any more.
Please send the rebased patches.

I have also noticed in the 'move_js_files_to_static_directory.patch', couple of modules have two definitions for 'module_use_template_javascript' method.
Also - 'module_paths.json' is auto generated, please don't include that in the patch.

-- Thanks, Ashesh 


--

Thanks & Regards,

Ashesh Vashi
EnterpriseDB INDIA: Enterprise PostgreSQL Company


http://www.linkedin.com/in/asheshvashi


On Tue, Jul 25, 2017 at 3:56 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi,

Please find rebased patches and review.

On Sat, Jul 22, 2017 at 1:03 AM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi,

Following are the changes in patches attached:

1) Move Javascripts from Templates directory to Static for following modules:
 - About
 - Browser nodes
 - Dashboard
 - FileManager
 - Vendor/snap.svg
 - Preferences
 - Settings
 - Backup
 - Datagrid
 - Debugger
 - Sqleditor
 - Grant Wizard
 - Import & Export
 - Maintenance
 - Restore and
 - User Management

2) Generate module JS path references dynamically/ Remove manually written paths.
Added a new file 'module_paths.json' which will update with absolute paths to module JS defined in every modules > __init__.py > 'def get_own_javascripts'.

When Flask service is started, it accesses the all module javascripts using `current_app.javascripts` which is called inside an route.
This call to route is made using Flask's `test_client` and thus those paths are written to `module_paths.json`(in javascript_bundler.py) which is used by webpack.shim.js file.

Three patches:
1. Move JS files to static directory
2. Deleted template JS
3. Generate JS paths before app starts

These patches needs to be committed separately as deleted files includes in the commit that makes difficult to look for changes through `git log <commit-has>`

Please review.

Thanks,
Surinder



On Thu, Jul 20, 2017 at 6:08 PM, Dave Page <dpage@pgadmin.org> wrote:
Thanks, applied.

On Thu, Jul 20, 2017 at 1:35 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi Dave,

Can you please review this patch please? I guess it was missed as i send 2 patches back to back in same email chain.
Please find inline detailed description of issue.

On Thu, Jul 20, 2017 at 12:37 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

The loading icon image is used in FileManager and it is still referenced from vendor `aciTree`(which is removed) so it gives 404 not found when FileManager is opened from Query tool/Backup utility.

Please find attached patch and review.

Thanks,
Surinder Kumar




--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company









Attachment

Re: [pgAdmin4]: Webpacking of static JS/CSS

From
Ashesh Vashi
Date:
On Thu, Jul 27, 2017 at 1:32 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi Ashesh,

I have removed changes which are not relevant to the moving file to static directory.
Committed the code for moving the javascript files from 'templates' to 'static' directory for most of the modules now.

I had to remove some redundant javascripts from the template directory manually.

-- Thanks, Ashesh 

Please find updated patch.

Thanks,
Surinder

On Thu, Jul 27, 2017 at 1:13 PM, Ashesh Vashi <ashesh.vashi@enterprisedb.com> wrote:
On Thu, Jul 27, 2017 at 12:57 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi Ashesh,

Please find attached patch 'move_js_files_static_dir' with --binary option.
As discussed, this patch should include changes related to moving the file to static directory only, and not related to the dynamic path generation.

-- Thanks, Ashesh 

Thanks,
Surinder

On Thu, Jul 27, 2017 at 12:09 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi Ashesh,

Changes in patch:

1. I have removed duplicate function 'module_use_template_javascript' from modules.
2. This patches only contains changes for moving template/js to static/js. I have excluded changes for making module paths dynamic as we will be using different approach to handle that.
3. Also, changed path references for JS modules to point to static/ dir instead of template/js in webpack.shim.js and webpack.config.js .

Please find attached patches

Thanks,
Surinder


On Thu, Jul 27, 2017 at 10:33 AM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Ok. I will send rebased patch.

On Thu, Jul 27, 2017 at 10:32 AM, Ashesh Vashi <ashesh.vashi@enterprisedb.com> wrote:


On Thu, Jul 27, 2017 at 10:22 AM, Ashesh Vashi <ashesh.vashi@enterprisedb.com> wrote:
Hi Surinder,

'delete_template_files.patch' is applied any more.
Please send the rebased patches.

I have also noticed in the 'move_js_files_to_static_directory.patch', couple of modules have two definitions for 'module_use_template_javascript' method.
Also - 'module_paths.json' is auto generated, please don't include that in the patch.

-- Thanks, Ashesh 


--

Thanks & Regards,

Ashesh Vashi
EnterpriseDB INDIA: Enterprise PostgreSQL Company


http://www.linkedin.com/in/asheshvashi


On Tue, Jul 25, 2017 at 3:56 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi,

Please find rebased patches and review.

On Sat, Jul 22, 2017 at 1:03 AM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi,

Following are the changes in patches attached:

1) Move Javascripts from Templates directory to Static for following modules:
 - About
 - Browser nodes
 - Dashboard
 - FileManager
 - Vendor/snap.svg
 - Preferences
 - Settings
 - Backup
 - Datagrid
 - Debugger
 - Sqleditor
 - Grant Wizard
 - Import & Export
 - Maintenance
 - Restore and
 - User Management

2) Generate module JS path references dynamically/ Remove manually written paths.
Added a new file 'module_paths.json' which will update with absolute paths to module JS defined in every modules > __init__.py > 'def get_own_javascripts'.

When Flask service is started, it accesses the all module javascripts using `current_app.javascripts` which is called inside an route.
This call to route is made using Flask's `test_client` and thus those paths are written to `module_paths.json`(in javascript_bundler.py) which is used by webpack.shim.js file.

Three patches:
1. Move JS files to static directory
2. Deleted template JS
3. Generate JS paths before app starts

These patches needs to be committed separately as deleted files includes in the commit that makes difficult to look for changes through `git log <commit-has>`

Please review.

Thanks,
Surinder



On Thu, Jul 20, 2017 at 6:08 PM, Dave Page <dpage@pgadmin.org> wrote:
Thanks, applied.

On Thu, Jul 20, 2017 at 1:35 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi Dave,

Can you please review this patch please? I guess it was missed as i send 2 patches back to back in same email chain.
Please find inline detailed description of issue.

On Thu, Jul 20, 2017 at 12:37 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

The loading icon image is used in FileManager and it is still referenced from vendor `aciTree`(which is removed) so it gives 404 not found when FileManager is opened from Query tool/Backup utility.

Please find attached patch and review.

Thanks,
Surinder Kumar




--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company