nway
Bundle commonjs javascript modules for the browser with cache optimisation: write browser-side script the node's way !
A Novadiscovery open-source project based on NodeJS
Github project: https://github.com/novadiscovery/nway ● Full documentation: http://nway.novadiscovery.com/
Features
- Transforms node.js modules to browser-side code
- Optimize client side cache by generating uniq, life-cachable, bundle of minimized sources.
- Provides a connect.js middleware to apply cache information to HTTP responses before the static provider treats the request (may be connect.static()).
- Asynchronous code splitter module to drive client bundle generation
- Module substitution to browser-only module
- Script compression using uglify
- Command line utility and JavaScript API
Installation
With npm
npm install -g nway
Or clone the git repository
git clone git://github.com/novadiscovery/nway.git
Then do
cd nwaynpm install
Link nway in npm to use the command line utility:
npm link
Introduction
It would be great if ...
When you use Node.js for server side code, it is really great to use the same language than client side for many reasons. But, when you create client side code, it would be even greater if you could organize your code the same way you do for server side. And, in some cases, be able to use the same code, server and client side (at least part of it).
CommonJS is from Mars, AMD is from Venus
Node provides an awesome way to create modular projects. This modularity, among other things, is a reason of the success of node.js. It uses the CommonJS convention, and is implementation is described in the module documentation.
But the CommonJS convention is a synchronous paradigm: all the dependencies must be instantly accessible. There is no mechanism to asynchronously load a dependency like Asynchronous Module Definition do. That is usualy what we need in rich client applications where source code size may be huge: to load only want we need and differ the loading of other parts of the application according to user needs.
nway magic trick
Nway transforms for you synchronous Node CommonJS modules into optimized client side AMD-like modules.
By detecting dependencies between your modules, and by adding an asynchronous code splitting concept, nway understands your application workflow and packs generated AMD-like modules in many cachable optimized files. This part of nway is inspired from Google Web Toolkit Code Splitting mechanism.
What nway does
Consider the following module graph. Each dot represents a CommonJS module eventually requiring some other modules. The entry point can be any module identified as a main module (usually the index.js of a package).
All your client side modules are standard Node.js modules, like any other module. They can require any server side module, packaged modules (such underscore or async) as well as Node.JS core modules (with some limitations).
The only difference is the use of an asynchronous require method (provided by nway). This asynchronous require arequire()
is used to identify splitting points in your application workfow.
Now, using nway, the following bundles will be identified, packed and optimized.
When a spliting point is reached, nway loads asynchronously the bundle that contains the required module (if not allready loaded) before returning the required module object.
The module & require point of view of the application workflow will be as follows (every behavior is configurable):
- The client loads an un-cachable bootstrap (
bootstrap.nocache.js
) that contains both the the basic loading system and the dependency map. - The bootstrap automaticaly loads the main (generated) bundle
E4FA2896C.js
: this is the one that contains the entry point module. - The entry point is started and, as the user goes deeper and deeper in the application, the spitting points are reached thus triggering the loading of the required bundles (C then D or B).
- The loaded bundle is a javascript file identified by a hash name (e.g.
5FA13642E.js
) based on its content and the configuration used by nway to compile it: Using nway's middleware or any other static server mechanism, the bundle's files are served with cache ad-vitam HTTP headers.
nway just use nodejs modules mechanism
As the main usage of nway is to transform any nodejs module into browser compatible modules, not any transformation of your code is required to make it work with nway.
Unless you need to explicitly split your application in asynchronously loaded bundles, nway only rely on standard module concept: module.exports
object and require()
mechanism:
- "In Node, files and modules are in one-to-one correspondence" (nodejs module documentation)
module.exports
is a object always defined in a module. By defaultmodule.exports
is an empty object{}
. The value assigned to themodule.exports
object is what a module expose to those who requiring this module.module.exports
support any kind of javascript value: object, string, number, function, date, boolean, array, etc.require()
is a module scoped function used to import other modules. The value returned byrequire('./foo.js')
is the value assigned tomodule.exports
by the./foo.js
module.- A required module is only executed once. The exported value is cached for later require() calls done in any other modules.
- The path passed to the
require()
function may be:- A relative or absolute path to a file (with, or without .js extension):
require('./foo')
wherefoo.js
is a file in the same folder. - A relative or absolute path to a folder with a
index.js
file inside:require('./bar')
wherebar
is a folder with aindex.js
file inside. - A node package installed with npm localy or globaly:
require('underscore')
whereunderscore
is node package resolvable by the nodejsrequire.resolver()
.
- A relative or absolute path to a file (with, or without .js extension):
Give us an exemple !
Let's first write some standard server-side node module.
The first exemple uses demo/01_simple:
demo/01_simple (where the commands, below, are executed) ├╴■ public | ├╴■ generated | └╴▫ index.html ├╴▫ simple.demo.js └╴■ src ├╴▫ bar.js ├╴▫ foo.js └╴▫ index.js
src/index.js: This is the entry point of our application
console.log('index.js: I require foo.js');var foo = require('./foo');console.log('index.js: I call run() on foo');foo.run();
src/foo.js:
console.log('foo.js: I require bar.js');var bar = require('./bar');console.log('foo.js: I add a function run() too my export object');exports.run = function() { console.log('foo.js: In my run(). Now I call bar.'); bar('Foo');}
src/bar.js:
console.log('bar.js: I export a function');module.exports = function(who) { console.log('bar.js: I say hello to', who);}
public/index.html: This file only import the nway generated bootstrap:
...<script src="./generated/bootstrap.nocache.js" type="text/javascript" charset="utf-8" sync="true"></script>...
This demo do really usefull things:
You can run node src/index.js
to see what happen when execute server side with node
- index.js require foo.js
- foo.js require bar.js
- bar.js export a Function
- foo.js add a function run to the exports object
- index.js call run() on foo.js
- foo.js call the function exported by bar in is run() function
- bar.js execute the executed function called by foo
nway it !
Now, using the nway command line, we generate the client version of this usefull application. The path given to nway is the main module (the entry point) of the application:
nway src/index.js
nway shows both the configuration used to generate client sources and the resolved dependency tree.
By default nway will output the client generated files in the public/generated
folder (you can change this by using the -o --output
option).
In the public/generated
there are now two files:
public/generated ├╴▫ bootstrap.nocache.js └╴▫ F9C09E355E2151A2.js
The bootstrap.nocache.js
is included by public/index.html
:
This tiny file contains a script loader that knows which file contains the required application modules. The bootstrap is very small, and changes as the application changes: that is why the client browser must never keep it in cache.
The F9C09E355E2151A2.js
file contains all the application modules':
The name may be different: the first 8 chars F9C09E35
change depending on nway's options and version number, the last 8 chars 5E2151A2
change with the modules contents: that is why the client browser can keep it in cache for life.
See nway middleware and cache optimisation for more information about this
Now, open public/index.html
in a browser. Open the javascript console. You should see the following output:
index.js: I require foo.jsfoo.js: I require bar.jsbar.js: I export a functionfoo.js: I add a function run() to my export objectindex.js: I call run() on foofoo.js: In my run(). Now I call bar.bar.js: I say hello to Foo
That is the same result of the server side execution using only node: node src/index.js
.
The demo/01_simple/simple.demo.js file do, with the nway API, exactly what we do with nway command line.
What happens when the index.html
file is executed?
- The
bootstrap.nocache.js
file is loaded, it contains the AMD mechanism (require, define, script loader, ...) and the application map (index of bundle files and the modules they contain). - When the DOM is ready, the bootstrap resolve wich file contains the main module — index.js, the entry point — (
F9C09E355E2151A2.js
) and loads it. - The
F9C09E355E2151A2.js
file register the modules it contains (among which the main module). - The main module is executed by the bootstrap with the appropriate scoped variables (require(), module.exports, etc.)
- The main module requires foo.js, foo.js is executed with some dedicated scoped require, and so on.
arequire: the asynchronous require()
Devide and conquer... speed
As we said before, nway does not force you to use nway-specific code to work: everything works just like it would in node. But, from the browser-side point of view, as your rich internet application become huge you don't want to load all the application at once. You just want to load progressively the parts of the application that the user needs. Also, if you update only a small part of your application, you don't want to loose all the benefits of the client browser cache (See nway middleware and cache optimisation)...
That is why, nway provides an application splitter: arequire()
.
arequire()
is totally compatible with node. It is just a node module that provides the same behavior than node's require()
except that the result (the exported value) is returned asynchronously to a callback function:
// The require() way:var foo = require('foo');console.log(foo);// The arequire() way:arequire('foo', function(err, foo) { console.log(foo)});
The goal of arequire()
is not to remplace require()
! You can just use it when you want nway to split your application.
How to import arequire()
arequire() is very special: it needs the require()
function of your module to work. So nway provides an arequire
generator:
// Get an arequire method for the current module:var arequire = require('nway/arequire')(require);
It is a good habit to import arequire() at the begining of your module. Of course, the variable name does not necessarily have to be arequire.
The only restriction is: do not use another variable with the same name in your module. It may work, but it could confuse nway's parser. In fact, essentialy due to performance reasons, nway's parser will not try to detect variable's scopes. Anyway you don't want to do this because is not neat!
Exemple using arequire(): demo/02_arequire
demo/02_arequire (where the commands, below, are executed) ├╴▫ arequire.demo.js ├╴■ public | ├╴■ generated | ├╴▫ index.html | └╴▫ style.css └╴■ src ├╴▫ bar.js ├╴▫ foo.js └╴▫ index.js
src/index.js: This is the entry point of our application
console.log('index.js: I get a arequire() function for my module');// This is where arequire is imported.//// It is a good habit to import arequire() on top of your module.// The variable name does not have to be 'arequire', but it can prevent// confusions to use this name.//// Caution: for parsing efficiency reason, do not use another variable with the same namevar arequire = require('nway/arequire')(require);console.log('index.js: For demo purposes, I export my application object');var application = module.exports = { goFoo: function(callback) { console.log('index.js: goFoo(). Asynchronously require foo.js. This is an application split point.'); arequire('./foo.js', function(err, foo) { console.log('index.js: foo.js is imported. Execute its exported function foo().'); foo(); callback && callback(); }) } ,goBar: function(callback) { console.log('index.js: goBar(). Asynchronously require bar.js. This is an application split point.'); arequire('./bar.js', function(err, bar) { console.log('index.js: bar.js is imported. Execute its exported function bar().'); bar(); callback && callback(); }) }}console.log('index.js: Go into the FOO part of the application:');application.goFoo(function() { console.log('index.js: Now go into the BAR part of the application:'); application.goBar();});
src/foo.js:
// foo.jsconsole.log("foo.js: I am a huge part of the application and I export a function");module.exports = function() { console.log("foo.js: I am executed");}
src/bar.js looks like foo.js
You can run node src/index.js
to see what happens when execute in server-side mode with node
index.js
creates anarequire()
for is module and then creates an application with some methods that asynchronously require other modulesindex.js
loads the FOO part of the application.- Once the FOO part is loaded,
index.js
loads the BAR part of the application
Run nway to generate the client version
Note: We do not specify index.js
, this is redundant in node since src
is a folder with an index.js
inside!
nway src
Open public/index.html
in your browser (open public/index.html
), the output in your javascript console is the same of the pure nodejs execution of our application: node src
:
index.js: I get a arequire() function for my moduleindex.js: For demo purpose, I export my application objectindex.js: Go in the FOO part of the application:index.js: goFoo(). Asynchronously require foo.js. This is an application split point.foo.js: I am a huge part of the application and I export a functionindex.js: foo.js is imported. Execute it exported function foo().foo.js: I am executedindex.js: Now go in the BAR part of the application:index.js: goBar(). Asynchronously require bar.js. This is an application split point.bar.js: I am a huge part of the application and I export a functionindex.js: bar.js is imported. Execute it exported function bar().bar.js: I am executed
Great but what is the difference ?
If you look at the public/generated
folder, there are 4 files:
public/generated ├╴▫ B70FBF6B67B040FD.js (bar.js) ├╴▫ B70FBF6B94EE0B24.js (foo.js) ├╴▫ B70FBF6BD1CB26E3.js (index.js and nway arequire generator) └╴▫ bootstrap.nocache.js
This is the nway splitting effect!
- One bundle contains the main module
index.js
but there is nothing else inside (except a very small nway module: the arequire function generator). - The two other bundles contains
foo.js
andbar.js
and are loaded only when the application needs them.
Asynchronously require many modules ?
Solution 1: pack your multi-dependency into a module
foo.js needs a.js and b.js to be asynchronously loaded before it can execute a function (the AMD way would be require(['a','b'], function (a, b) {})
)
Create a bundle_ab.js that exports a.js and b.js:
module.exports = { a: require('./a'), b: require('./b')};
Now in foo.js you can do:
arequire('./bundle_ab', function(error, results) { // Now you have results.a and results.b ...})
Solution 2: just do it async !
Many javascript libraries such as async.js or queue.js simplify this asynchronous pattern: You control exactly what you do and you can re-use the usefull patterns of this library in your application without the need to reload it.
foo.js with async:
var async = require('async'), arequire = require('nway/arequire')(require);async.parallel([ function(done) { arequire('./a.js', done)} ,function(done) { arequire('./b.js', done)}], function(results) { // Now you have results.a and results.b ...})
Module substitution
Some times you need to substitute a server-only module by a browser-compatible module.
There are many reasons to prevent yourself from doing this: even for testing where using headless browser is a much better solution to test browser-only modules.
In some cases you have no choice: for example, if you are using an external package that require a server-side-only module that is easy to override for browser side.
See module substitution sample in substitute.demo.js
Cache optimisation and nway middleware()
nway's cache optimisation is based on:
- nway middleware. It applies http cache optimisation on nway-generated content.
- Generated bundle-file naming. A hash-name is used based on: nway version, compilation options and bundle content.
- A very tiny bootstrap script, never cached, that drives bundle loading
Exemple: consider the following public/generated
folder:
public/generated ├╴▫ bootstrap.nocache.js ├╴▫ CDC551BB0420A359.js ├╴▫ CDC551BB18A726CD.js ├╴▫ CDC551BB1A74CB97.js ├╴▫ CDC551BB29E43756.js ├╴▫ CDC551BB5FCDA7AD.js └╴▫ CDC551BB822417A8.js
bootstrap.nocache.js
is the only file explicitly included in you web page:
This file contains a script loader that knows which file contains the required application modules. The bootstrap is very small, and change as the application changes: that is why the client browser must never keep it in cache.
Files like CDC551BB0420A359.js
contains the application modules:
The first 8 chars CDC551BB
change depending on nway's options and version number: that is why many files begin with this string. The last 8 chars 0420A359
change with the modules contents.
To summarize:
- The bootstrap must never be cached (or, at least, it can be a http 304 response).
- The bundles are naturally cachable for life.
The nway middleware does exactly that: it explicitly forces the http header obtain this effect (caching all generated bundles by the bootstrap). But you could do the same with any http server.
Exemple with a small connect.js driven http server:
var connect = require('connect') , http = require('http') , nway = require('nway') ; // The options consumed by nway.middleware() // must be the same than the ones used for compilation // (at least the 'bootstrap', 'client' and 'extension' keys) var options = {client:'/generated', bootstrap: 'bootstrap.nocache.js', extension: '.js'}; var app = connect() .use(nway.middleware(options)) .use(connect.static('public')) // Path to public files .listen(3000); http.createServer(app).listen(3000);
Your app is now served on http://localhost:3000/
Limitations
Dynamic require & arequire module argument is not yet allowed (may be one day... or may be not):
// You can't yet do this kind of things:var foobar = 'foobar';require(foobar);// Or this:var x = 'foo';require('module_' + x);// Or this:['a','b'].forEach(function(m) { require(m) });// Or this:async.map(['a','b'], arequire, function(err, results) {});
Variable re-affecting for require or arequire:
// You can't do this:var arequire = require('nway/arequire')(require);var split = arequire;split('./a.js', function() {}) // Ok in nodejs but this won't work on browser side !// Nor this:var r = require;r('path'); // Ok in nodejs but this won't work on browser side !
Using server-only package:
- nway can use alternative to node builtin packages (
event
,path
, etc.). However some packages, such as nodefs
, can only be executed on server side (nway is compatible with browserify packages). - All packages that uses C++ compiled modules... can't be compiled by nway to be run in the browser.
Anyway, when you experience this kind of needs, consider reviewing in your application design. If after that your needs persist, consider using options.substitute
to provide a browser-alternative module.
Command line
Usage:
$ nway --helpUsage: nway [options] <index>Options: -h, --help output usage information -V, --version output the version number -b, --bootstrap [string] Bootstrap file path [bootstrap.nocache.js] -c, --client [string] Client relative path to load module packages [./generated] -e, --extension [string] Extension to append to generate package [.js] -f, --force Force compilation, even if the destination file exist@[false] -m, --nomangle Do not mangle names (used only with --uglify) [false] -o, --output [string] Output path for generated module packages [./public/generated] -p, --prepend [string] Prepend all generated script with a string [null] -n, --norun Do not run entry point automaticaly when DOM is ready [false] -s, --nosqueeze No deep optimization (used only with --uglify) [false] -u, --uglify Uglify source using Uglify.js [false] -y, --beautify Generate human readable source (used only with --uglify) [false] --onepack Pack all modules in a standalone source and send the result on the standards output [false]
ZSH Completion
To enable zsh completion add this in any zsh configuration file: compdef _gnu_generic nway
Play with command line & demo
Go in the demo folder:
cd demo
Execute nway without options, just choose the application main module as entry point:
nway src/index.js
Now in the public/generated
folder contain the files generated by nway:
public/generated ├╴▫ 15986D7A18A726CD.js ├╴▫ 15986D7A214696ED.js ├╴▫ 15986D7A822417A8.js ├╴▫ 15986D7A8ECD6654.js ├╴▫ 15986D7AF2AEA598.js ├╴▫ 15986D7AF395D729.js └╴▫ bootstrap.nocache.js
Every time you change a file content, or any nway options, the generated file names change.
Then open the demo in a navigator:
open public/index.html
Now, you can play with the demo application:
- Click on buttons to start a part of the application
- In the debug output (or your browser console) you can see the bundles loaded by nway.
You can do more things by using your browser javascript console (developper tool):
- Get the entry point:
var ep = require('/')
- Then play with your entry point object:
ep.voronoi()
,ep.load('underscore', function(_) {alert(_)})
, ...
Debug information
nway uses debug to output some debug information.
To show debug information, add DEBUG='nway*'
before any command or script that uses nway:
DEBUG=nway* nway src/index.js -f
(-f
is used to force nway to re-generate all files)
Play with some command line options
-u, --uglify & co
With the option uglify
, the generated sources are optimized and minimized with UglifyJS:
nway src/index.js -u
If you look at the generated sources in public/generated
, you can see that all the sources are deeply optimized.
Generating sources using uglify takes much much longer (at least the first time). Consider using this optimization for production compilation only.
Some other options can alter uglify behaviours:
-m, --nomangle
: Keep original variable and function names-s, --nosqueeze
: Disable deep source optimizations process-y, --beautify
: Add indentation and line return to make generated source human readable.
-p, --prepend [string]
Prepends all generated files with the given string (such a copyright notice):
nway src/index.js -p '// Copyright Foobar'
--onepack
Forces nway to bundle all the generated bundles in one single bundle. When you use the --onepack
options, nway do not write anything to the disk, instead it writes the compilation result to the standard output.
This is usefull when you do not want a bootstrap file and you do not care of application splitting (to dump the script in a standalone html file for instance).
nway src/index.js --onepack > mypack.js
JavaScript API
This is the common usage of nway. For details, please read the API documentation.
var nway = require('nway');nway({ // Generated file destination (absolute or relative path // to the current working directory) output : './public/generated' // Client url to the generated files (relative or absolute url) , client : './generated' // The bootstrap file name (*.nocache.* pattern may be used to force static // file server to disable cache) , bootstrap : 'bootstrap.nocache.js' // Globals are used for script optimisation : those variable // are scoped in each bundle to reduce file size when using // uglify to mangle variable names , globals : 'window, document, console, require' // The entry point (absolute or relative path to the // current working directory) , index : './index.js' // Generated packed file extension , extension : '.js' // Do not automaticaly run main entry point (the index) when the DOM is ready // (you have to do require('/') by your self when the dom is ready) , norun : false // Optimise source with uglify , uglify : false // Uglify option : do not mangle names , nomangle : false // Uglify option : do not do deep source optimisation , nosqueeze : false // Uglify option : generate readable source (comments are // allways removed with uglify) , beautify : false // Force re-generation even if a generated file allready exist with // the same hash , force : false // Prepend all the generated file with this string (maybe a copyright) , prepend : null // core & node_modules remplacements // // Node.js core & node_modules remplacement are resolved using the following process : // // - Check for an alternative in options.alternative : { 'package-name': 'alternative-package-name'} // - Check for a browserify alternative : {'http': 'node_modules/http-browserify'} // - Check for a nway alternative : {'http': 'node_modules/http-nway'} // , alternative : {} // Replace a file path resolved by another file path path , substitute : {} // Alias list : // // keys = module alias // value = module resolvable filepath // // nway provide a default alias for main module (entry point) : '/' // // You may define alias to manualy doing a require('myAlias') a module // in the browser. Remember nway hide the real module path in generated sources // as long as you not explicitly provide an alias to them. , alias : {} // Used to force re-generation of bundle files when nway version // has change (as the options are used to generate global uniq id) , version : require('../package.json').version // To change global uniq id , catkeyboard : '' // Array of patterns of filepath to exclude form parsing // Use it to enhance compilation speed on allready bundled source without // any commonjs mechanism inside. // // Allowed values are : // - string : Used as a minimatch pattern on absolute file path // - regexp : Used to test absolute file path // - function : Receive the absolute file path and the Module object. Returns a boolean. , noparse : [] // Exclude some dependency from the generation process // You will have to add those dependencies by your self using require.define() // Exclude is an array of minimatch (https://github.com/isaacs/minimatch) wildcards , excludes : [] // compress is a compression function. // default is nway.defaultCompressor (based on uglify) , compress : null // prewrite is a function to execute on the source before // write to disk : the function receive a source, and an object // The object may be (instanceof) : an nway/lib/Bootstrap or a nway/lib/DepNode (bundle) // This function MUST always return a source , prewrite : null // Use the onepack builder : all the modules bundled in one source (ignore async splitter) , onepack : false // Parsers : // an hash of extra parser objects : // // - match : minimatch pattern or regex on module filepath or function (with the same // arguments passed to the parse function listed below) // - parse : Parser function that return the parsed (and transformed) source // (see nway/lib/parsers/*) for exemples // // The parse function used in main nway parser is the one with a `match` // pattern that suceed on the filepath // // Each `parse` function receive those arguments : // // - src (string) : Source to parse and transform // - module (object) : Module object created by the main parser function // - makeModule (function): Main parser module creator : receives an absolute // filepath to parse, and return a new Module object. // - options (object): An nway options object // // And return a parsed (and some time transformed) source // // nway defaults parsers (in nway/lib/parsers) : // // - commonjs javascript source (.js) // - json source (comments allowed) (.json) // , parsers : []})
License
(The MIT License)
Copyright (c) 2012-2013 Novadiscovery osproject@novadiscovery.com
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the 'Software'), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
Bootstrap.js
lib/Bootstrap.jsAPI Summary
- Bootstrap()
Bootstrap constructor
- Bootstrap#generate()
Generate the bootstrap source
- Bootstrap#makePath()
Generate the packet filepath
- Bootstrap#write()
Write the packet in the file system
Bootstrap#generate(tree, options, options, options) public
Generate the bootstrap source
Bootstrap.prototype.generate = function(tree, options, template, data) { var alias = {} , packets = tree.getAllBundles(options) , modules = tree.getAllModules() , config , src , template = template || templates.bootstrap , data = data || {} ; // The tree must, at least, contain one module : if(!modules[0]) { throw new Error("The tree must contain, at least, one module"); } // Create a config information for bootstrap config = { client : options.client ,extension : options.extension ,norun : options.norun ,map : {} ,main : modules[0].uid // Root module } // Packet id mapping _.each(packets, function(packet) { config.map[packet.getUID()] = _.pluck(packet.modules, 'uid'); }); // Transposed alias list : _.each(options.alias, function(to, from) { var realpath; try { realpath = fs.realpathSync(to); } catch(e) { debug('Invalid alias (no file found) (1): ' + from + ' -> ' + to); return; } // Search for related module : var module = _.find(modules, function(m) { return m.path == realpath }); if(!module) { debug('Invalid alias (no module found) (2): ' + from + ' -> ' + to); } else { alias[from] = module.uid; } }); // Always add alias / to entry point : alias['/'] = modules[0].uid; // Template data : var data = _.extend({}, { config : JSON.stringify(config) ,alias : JSON.stringify(alias) ,loader: templates.loader }, data); // Apply to template : src = template.replace(/\{\{(\w*)\}\}/g, function(all, match) { return data[match] || ''; }); this.fromSize = src.length; // compress ? if(options.compress) { debug(' - compress : '); src = options.compress(src, options); } if('function' == typeof options.prewrite) { src = options.prewrite(src, this); } this.toSize = src.length; return src;}
Bootstrap#makePath(options) public
Generate the packet filepath
Bootstrap.prototype.makePath = function(options) { return join(options.output, options.bootstrap);}
Bootstrap#write(tree, options, callback) public
Write the packet in the file system
Bootstrap.prototype.write = function(tree, options, callback) { var self = this , path = this.makePath(options) , src = this.generate(tree, options) , from = this.fromSize , to = this.toSize ; debug(' - write', sprintf("%'.-30s %7s > %7s (%s%%)", 'bootstrap : ' + path, bytesToSize(from), bytesToSize(to), Math.round(to * 100 / from))); // Make sure path exists, then write file : mkdirp(dirname(path), function() { // Write file : fs.writeFile(path, src, function(err) { if(err) return callback(err); callback(); }); });}
DepNode.js
lib/DepNode.jsAPI Summary
- DepNode()
DepNode constructor
- DepNode#removeChild()
Remove a child node
- DepNode#appendChild()
Add a child node to this node
- DepNode#hasModule()
@param {string} module_id Module id
- DepNode#removeModule()
Remove a module object
- DepNode#addModule()
Add a module
- DepNode#getBranchModules()
Return all descendant modules of this node and is descendants
- DepNode#removeDescendantDuplicate()
Remove in children any modules required in the current node
as they will be allready defined before - DepNode#getDuplicates()
Return module that appear more than once in the given
list of nodes and their childs - DepNode#turnUpChildDuplicate()
Turn up to the first common ancestor any module
required into more than one descendant node - DepNode#removeEmptyNodes()
Remove from the DepNode tree the nodes that do not contain
any modules - DepNode#optimise()
Run optimisation :
- DepNode#getUID()
Generate a uniq id for the current node base
on the uniq ids of the module owned by this node - DepNode#getModuleIDs()
Get an array with modules id for this node
- DepNode#toBundle()
Return a packet object for the current node
- DepNode#getAllBundles()
Return an array of all packets for the current
tree - DepNode#getModules()
Return an array of the current node Modules objects
- DepNode#getAllModules()
Return an array of the current node Modules objects
- DepNode#generate()
Generate the packet source
- DepNode#makePath()
Generate the packet filepath
- DepNode#write()
Write the packet in the file system
- DepNode#toColoredXML()
Return the dependency tree in colored XML string (for information
purpose)
DepNode(id, nway) public
DepNode constructor
A simple DOM to manage code dependencies
function DepNode(id, options) { debug('new DepNode(%s)', id); if('string' != typeof id) { throw new Error('Invalid argument : id must be a string'); } checkOptions(options); // Parent DepNode reference this.parent = null; // Child nodes array this.children = []; // This node uniq id this.id = id; // List of the modules uniq id stored // owned by this node this.modules = []; // Current nway options (stored for uid generation) this.options = options // Used by debug to show optimization ratio this.fromSize = 0; this.toSize = 0;};
DepNode#removeChild(node) public
Remove a child node
DepNode.prototype.removeChild = function(node) { if(node.parent !== this) throw new Error('Node not found'); node.parent = null; this.children = _.without(this.children, node); return node;};
DepNode#appendChild(node) public
Add a child node to this node
DepNode.prototype.appendChild = function(node) { if(node.parent === this) return this; if( node.parent ) { node.parent.removeChild(node); } node.parent = this; this.children.push(node); return this;};
DepNode#hasModule() public
@param {string} module_id Module id
DepNode.prototype.hasModule = function(module_id) { return _.any(this.modules, function(module) { return module.uid == module_id; });};
DepNode#removeModule(module_id, [recursive]) public
Remove a module object
DepNode.prototype.removeModule = function(module_id, recursive) { this.modules = _.filter(this.modules, function(module) { return module.uid != module_id; }); if(recursive) { _.invoke(this.children, 'removeModule', module_id, true); }};
DepNode#addModule(module_id) public
Add a module
DepNode.prototype.addModule = function(module) { if(!this.hasModule(module.uid)) { this.modules.push(module); } return true;};
DepNode#getBranchModules() public
Return all descendant modules of this node and is descendants
DepNode.prototype.getBranchModules = function() { return _.union( [], this.modules, _.flatten(_.map( this.children, function(node) { return node.getBranchModules() })));}
DepNode#removeDescendantDuplicate() public
Remove in children any modules required in the current node
as they will be allready defined before
DepNode.prototype.removeDescendantDuplicate = function() { var me = this; _.each(this.children, function(node) { node.removeDescendantDuplicate(); _.each(me.modules, function(module) { node.removeModule(module.uid, true); }); });}
DepNode#getDuplicates(branches) public
Return module that appear more than once in the given
list of nodes and their childs
DepNode.prototype.getDuplicates = function(branches) { var count = {}; var result = []; _.each(_.flatten(branches), function(module){ if(!count[module.uid]) { count[module.uid] = 0; } count[module.uid]++; if(count[module.uid]>1) result.push(module); }) result = _.uniq(result); return result;}
DepNode#turnUpChildDuplicate() public
Turn up to the first common ancestor any module
required into more than one descendant node
DepNode.prototype.turnUpChildDuplicate = function() { var me = this; // Run on child nodes before : _.invoke(this.children, 'turnUpChildDuplicate'); // Check the union of all child nodes modules : var child_modules = _.invoke(this.children, 'getBranchModules'); var duplicates = this.getDuplicates(child_modules); // Remove module intersection (referenced, at least, in two childrens) // and move dependency to current node : _.each(duplicates, function (module) { // Remove descendant _.invoke(me.children, 'removeModule', module.uid, true); // Add to current node : me.addModule(module); });}
DepNode#removeEmptyNodes() public
Remove from the DepNode tree the nodes that do not contain
any modules
DepNode.prototype.removeEmptyNodes = function() { var me = this; _.each(this.children, function(child) { if(!child.modules.length && !child.children.length) { me.removeChild(child); } else { child.removeEmptyNodes(); } })}
DepNode#optimise() public
Run optimisation :
- Remove dependencies to modules allready required upper in the tree (removeDescendantDuplicate())
- Put module required more than once time in the dependency tree in the common ancestor node (runUpChildDuplicate())
- Remove empty nodes that do not contain any module after the previous optimisations
DepNode.prototype.optimise = function() { this.removeDescendantDuplicate(); this.turnUpChildDuplicate(); this.removeEmptyNodes();};
DepNode#getUID() public
Generate a uniq id for the current node base
on the uniq ids of the module owned by this node
DepNode.prototype.getUID = function() { var guid = getUID(this.options); return guid + getUID(_.pluck(this.modules,'uid').join('-'));};
DepNode#getModuleIDs() public
Get an array with modules id for this node
DepNode.prototype.getModuleIDs = function() { return _.pluck(this.modules, 'uid');}
DepNode#toBundle() public
Return a packet object for the current node
DepNode.prototype.toBundle = function() { return this;}
DepNode#getAllBundles() public
Return an array of all packets for the current
tree
DepNode.prototype.getAllBundles = function() { var packets = []; var walk = function(node) { packets.push(node.toBundle()); _.each(node.children, function(child) { walk(child); }); } walk(this); return packets;}
DepNode#getModules() public
Return an array of the current node Modules objects
DepNode.prototype.getModules = function() { return this.modules;}
DepNode#getAllModules() public
Return an array of the current node Modules objects
DepNode.prototype.getAllModules = function() { var modules = []; var walk = function(node) { _.each(node.getModules(), function (module) { modules.push(module); }); _.each(node.children, function(child) { walk(child); }); } walk(this); return modules;}
DepNode#generate(options) public
Generate the packet source
DepNode.prototype.generate = function(options) { debug('Generate %s', this.uid); var src = ''; // Concat sources of the packet _.each(this.modules, function(module) { debug(' -', module.relpath) var moduleSrc = module.generate(); if('function' == typeof options.prewrite) { moduleSrc = options.prewrite(moduleSrc, module); } src += moduleSrc; }); // Template substitution data var data = { body : src ,globals : options.globals } // Add to template : src = templates.packet.replace(/\{\{(body|globals)\}\}/g, function(all, match) { return data[match]; }); this.fromSize = src.length; // compress ? if(options.compress) { debug(' - compress : '); src = options.compress(src, options); } // Prepend ? if(options.prepend) { src = options.prepend + '\n' + src; } if('function' == typeof options.prewrite) { src = options.prewrite(src, this); } this.toSize = src.length; return src;}
DepNode#makePath(options) public
Generate the packet filepath
DepNode.prototype.makePath = function(options) { return join(options.output, this.getUID() + options.extension);}
DepNode#write(options, callback) public
Write the packet in the file system
DepNode.prototype.write = function(options, callback) { var self = this , path = this.makePath(options) , src = this.generate(options) , from = this.fromSize , to = this.toSize ; debug(' - write', sprintf("%'.-30s %7s > %7s (%s%%)", 'packet : ' + path, bytesToSize(from), bytesToSize(to), Math.round(to * 100 / from))); // Write file : fs.writeFile(path, src, function(err) { if(err) return callback(err); callback(); });}
DepNode#toColoredXML() public
Return the dependency tree in colored XML string (for information
purpose)
DepNode.prototype.toColoredXML = function() { var deep = arguments[0] || 0 , packet = this.toBundle() , modules = this.modules , children = this.children , s = '' , i = repeat(' ', 4) , i1 = repeat(i, deep + 1) , i2 = repeat(i, deep + 2) , i3 = repeat(i, deep + 3) , i4 = repeat(i, deep + 4) ; if(deep === 0) { s += '<?xml version="1.0" encoding="UTF-8"?>\n'.grey; s += '<tree>\n'.grey; } s += i1 + '<packet id="'.grey + this.getUID().bold.blue + '" '.grey + 'output="' .grey + packet.makePath(this.options) + '" '.grey; if(packet.fromSize) { s += 'from="' .grey + packet.fromSize + '" '.grey + 'to="' .grey + packet.toSize + '" '.grey + 'ratio="'.grey + Math.round(packet.toSize * 100 / packet.fromSize) + '%'+'" '.grey; } s += '>\n'.grey; s += i2 + '<modules>'.grey + (modules.length ? '\n' : ''); _.each(modules, function(module) { s += i3 + '<module id="'.grey + module.uid.bold.blue + '" '.grey + 'file="'.grey + module.relpath + '" '.grey + 'isNodeModule="'.grey + module.isNodeModule + '" '.grey + 'isCore="'.grey + module.isCore + '"'.grey + (module.req.length ? '>\n' : '/>\n').grey if(module.req.length) { _.each(module.req, function(req) { s += i4 + '<require id="'.grey + req.uid + '" async="'.grey + req.async +'"/>\n'.grey }) } s += (module.req.length ? i3 + '</module>\n'.grey : ''); }); s += (modules.length ? i2 + '' : '') + '</modules>\n'.grey; if(children.length) { s += i2 + '<packets>'.grey + (children.length ? '\n' : ''); _.each(children, function(child) { s += child.toColoredXML(deep + 3); }); s += (children.length ? i2 + '' : '') + '</packets>\n'.grey; } s += i1 + ('</packet>\n'.grey); if(deep === 0) { s += '</tree>\n'.grey; } return s;};
Module.js : The module object
lib/Module.jsModule are created by the parser and contain all the required informations
for the compilation
API Summary
- Module()
Module constructor
- Module#getRequired()
Get an uid array of required module
- Module#generate()
Generate the pseudo-amd source for the current module
Module(init) public
Module constructor
function Module(init) { var self = this; // A uniq id this.uid = null; // List of required modules : // a list of object with to property : // - uid : the required module uid // - async : the require is synchronous or asynchronous this.req = []; // Absolute real path to the module file this.path = null // Relative path to the module (relative to the process.cwd()) this.relpath = null // Flag : this is a module located in node_modules this.isNodeModule = false // Flag : this is a nodejs core module this.isCore = false; // Source : the module script source this.source = null; // Prevent from parsing for require() calls: this.doNotParse = false; _.each(init, function(v,k) { self[k] = v});}
Module#getRequired([async]) public
Get an uid array of required module
Module.prototype.getRequired = function(async) { var list = _.filter(this.req, function(m) { return async === true ? m.async : async === false ? !m.async : true; }); return _.pluck(list, 'uid');}
Module#generate() public
Generate the pseudo-amd source for the current module
Module.prototype.generate = function() { var data, src; // Template substitution data data = { body : this.source ,uid : this.uid ,path: this.relpath } // Apply to template and return generated source : return templates.define.replace(/\{\{(body|uid|path)\}\}/g, function(all, match) { return data[match]; });}
arequire.js : Async require, an nway splitter
lib/arequire.jsAPI Summary
- arequireGenerator()
Generate an asynchronous require function based
on a module require function
arequireGenerator(parentRequire) public
Generate an asynchronous require function based
on a module require function
Usage :
var arequire = require('nway/arequire')(require)arequire('mymodule', function(mymodule) { console.log(mymodule) })
function arequireGenerator(parentRequire) { if('function' != typeof parentRequire) { throw new Error('Invalid parentRequire argument : You must provide a node require function. You may have forgotten to generate your arequire function like this : var arequire = req'+'uire("nway/arequire")(require)') } return function(required, callback) { if('string' != typeof required) { throw new Error('Invalid module path : You must provide a valid module path.') } if('function' != typeof callback) { throw new Error('Invalid callback argument : You must provide a function') } function cb(err) { if(err) { return callback(err); } try { callback(null, parentRequire(required)); } catch(e) { return callback(e); } } if(require.loader) { // Client side require.loader.load(required, cb); } else { // Server side process.nextTick(cb); } }}
buildTree.js
lib/buildTree.jsAPI Summary
- buildTree()
Create the dependency tree weighted from a given
module uniq id used as entry point (aliased '/' later) - findModule()
Find a module by is uniq id
- builder()
Build a module dependency tree
buildTree(options, modules, [optimise]) public
Create the dependency tree weighted from a given
module uniq id used as entry point (aliased '/' later)
The tree is composed of DepNode object.
function buildTree (options, modules, optimise) { var packageId = 1 , optimise = optimise === false ? false : true; debug('start buildTree()'); debug(' optimise: %s', optimise); debug(' count modules: %s', modules.length);
findModule(uid) public
Find a module by is uniq id
function findModule(uid) { return _.find(modules, function(module) {return module.uid === uid} ); }
builder(fromUID, [node]) public
Build a module dependency tree
function builder(fromUID, node) { debug('builder fromUID:%s',fromUID); // Create a node (or use the provided one) var node = node || new DepNode((packageId++).toString(), options); // Get the entry point module object var fromModule = findModule(fromUID); // Add entry point to the module node.addModule(fromModule); debug(' fromModule.req.length:%s', fromModule.req.length); // For each module dependency, add module to // the node _.each(fromModule.req, function(link) { var module = findModule(link.uid); // Duplicate : pass if(node.hasModule(module.uid)) return; if(link.async) { debug(' Add async module (%s) to node %s', link.uid, node.id); // Async : Create a child node node.appendChild(builder(link.uid)); } else { debug(' Add sync module (%s) to node %s', link.uid, node.id); // Sync : add module node.addModule(module); builder(link.uid, node); } }); return node; } // Main recursive call var tree = builder(modules[0].uid); // Optimisation (see DepNode about optimisation) if(optimise) tree.optimise(); return tree;}
builder.js
lib/builder.jsAPI Summary
- builder()
The builder
builder(options, callback) public
The builder
This is the main nway function. The builder read the options
object and generate all the static files.
function builder(options, callback) { // Default compressor ? if(options.uglify && !options.compress) { options.compress = defaultCompressor; } if(options.onepack) { return onepack(options, callback) } else { return standard(options, callback) }}
checkOptions.js
lib/checkOptions.jsAPI Summary
public
Throw an error if the options object is
not a valid nway options object (key check)
Used by deep nway object that require a valid
nway object. As public API (such nway()) always extend options
passed with user with default object this is essentialy
for unit testing constraint and deep API usages.
module.exports = function(options, mustHaveKeys) { var mustHaveKeys = mustHaveKeys || _.keys(defaults) , intersection = _.intersection(mustHaveKeys, _.keys(options)) ; if( ('object' !== typeof options) || (intersection.length !== mustHaveKeys.length) ) { throw new Error('Invalid nway object'); }}
nway command line executable
lib/cmd.js// IMPORTvar Command = require('commander').Command , colors = require('colors') , fs = require('fs') , _ = require('underscore') , sprintf = require('underscore.string').sprintf , nway = require('./nway') , defaultOptions = require('./defaultOptions') , indexResolve = require('./indexResolve');;// Create a command line utilityvar app = new Command('nway');app .version(nway.version) .usage('[options] <index>') .option('-b, --bootstrap [string]', 'Bootstrap file path ['+defaultOptions.bootstrap+']', defaultOptions.bootstrap) .option('-c, --client [string]', 'Client relative path to load module packages ['+defaultOptions.client+']', defaultOptions.client) .option('-e, --extension [string]', 'Extension to append to generate package ['+defaultOptions.extension+']', defaultOptions.extension) .option('-f, --force', 'Force compilation, even if the destination file exist@['+defaultOptions.force+']', defaultOptions.force) .option('-m, --nomangle', 'Do not mangle names (used only with --uglify) ['+defaultOptions.nomangle+']', defaultOptions.nomangle) .option('-o, --output [string]', 'Output path for generated module packages ['+defaultOptions.output+']', defaultOptions.output) .option('-p, --prepend [string]', 'Prepend all generated script with a string ['+defaultOptions.prepend+']', defaultOptions.prepend) .option('-n, --norun', 'Do not run entry point automaticaly when DOM is ready ['+defaultOptions.norun+']', defaultOptions.norun) .option('-s, --nosqueeze', 'No deep optimization (used only with --uglify) ['+defaultOptions.nosqueeze+']', defaultOptions.nosqueeze) .option('-u, --uglify', 'Uglify source using Uglify.js ['+defaultOptions.uglify+']', defaultOptions.uglify) .option('-y, --beautify', 'Generate human readable source (used only with --uglify) ['+defaultOptions.beautify+']', defaultOptions.beautify) .option('--onepack', 'Pack all modules in a standalone source and send the result on the standards output ['+defaultOptions.onepack+']', defaultOptions.onepack);// Parse the command line argumentsapp.parse(process.argv);// Retrieve the entry pointvar index = app.args[0] ? app.args[0] : null;if(!index) { console.log('Error : '.bold.red, 'Invalid arguments. Index required.'); console.log(app.helpInformation()); process.exit(1);}// Resolve index :var resolvedIndex = indexResolve(index)//= resolvedIndex;try { if(!resolvedIndex) throw new Error('Index argument "' + index + '" can\'t be resolved to a valid entry point.');} catch(e) { console.log('Error : '.bold.red + ' %s', e.message); process.exit(1);}app.index = resolvedIndex;// Prepare the configuration object :var config = _.extend({}, defaultOptions);_.each(config, function(v, k) { if(app[k] && (k != 'version')) { config[k] = app[k]; }});if(config.onepack) { nway(config, function(err, result) { if(err) { console.log('Error : '.bold.red, err ); process.exit(1); } else { console.log(result); } });} else { console.log(''); console.log(' Start nway Generator '.blue.inverse); // Print out the current configuration console.log('--------------------- Configuration -------------------'.blue.bold); _.each(config, function(v, k) { console.log(sprintf("%'.-30s %s", k.blue + ' ', v)); }); // Start the generator nway(config , function(err, result) { if(err) { console.log('Error : '.bold.red, err ); process.exit(1); } else { console.log('-------------------- Dependency tree ------------------'.blue.bold); console.log(result.tree.toColoredXML()); console.log(' Generation succesfull '.blue.inverse); console.log(''); } });}
decojs.js
lib/decojs.jsAPI Summary
- decojs()
Remove comment in a source code
decojs(str) public
Remove comment in a source code
decojs() is remove javascript style single line
and multiline comments (// and /*)
function decojs(str) { var i , curChar, nextChar, lastNoSpaceChar , inString = false , inComment = false , inRegex = false , newStr = '' , stringOpenWith ; for (i = 0; i < str.length; ++i) { curChar = str.charAt(i); nextChar = str.charAt(i + 1); // In string switcher if (!inRegex && !inComment && (curChar === '"' || curChar === "'") && str.charAt(i - 1) !== '\\') { if(inString && (curChar === stringOpenWith)) { inString = false; stringOpenWith = null; } else if(!inString) { inString = true; stringOpenWith = curChar; } } // In regex switcher if((!inComment && !inString) && (curChar === '/')) { if(inRegex // Not escaped ... /myregexp\/...../ && (str.charAt(i - 1) !== '\\') // Or escape char, previously escaped /myregexp\\/ || ((str.charAt(i - 1) === '\\') && (str.charAt(i - 2) === '\\'))) { inRegex = false; } else { if(~['=',',','('].indexOf(lastNoSpaceChar)) { inRegex = true; } } } if(!~['', ' '].indexOf(curChar)) { lastNoSpaceChar = curChar; } // we are not inside of a string or a regex if (!inString && !inRegex) { // singleline comment start if (!inComment && curChar + nextChar === '/'+'/') { ++i; inComment = 1; // singleline comment end } else if (inComment === 1 && curChar === '\n') { inComment = false; // multiline comment start } else if (!inComment && curChar + nextChar === '/'+'*') { ++i; inComment = 2; curChar = ''; // multiline comment end } else if (inComment === 2 && curChar + nextChar === '*'+'/') { ++i; inComment = false; curChar = ''; } if (inComment === 2 && curChar === '\n') { // curChar = '\n' (keep line return) } else if (inComment) { curChar = ''; } } newStr += curChar; } return newStr;}decojs.version = require('../package.json').version;
nway default compressor (uglify)
lib/defaultCompressor.jsAPI Summary
public
Default compressor
module.exports = function(src, options) { try { var ast = jsp.parse(src, false, true); if(!options.nomangle) { ast = pro.ast_mangle(ast); } if(!options.nosqueeze) { ast = pro.ast_squeeze(ast); } src = pro.gen_code(ast, { beautify: options.beautify }); } catch(e) { debug('Compression error : source returned without compression.', e) } return src;}
defaultOptions.js : The default option object
lib/defaultOptions.jsDefault option object used by nway
module.exports = { // Generated file destination (absolute or relative path // to the current working directory) output : './public/generated' // Client url to the generated files (relative or absolute url) , client : './generated' // The bootstrap file name (*.nocache.* pattern may be used to force static // file server to disable cache) , bootstrap : 'bootstrap.nocache.js' // Globals are used for script optimisation : those variable // are scoped in each packet to reduce file size when using // uglify to mangle variable names , globals : 'window, document, console, require' // The entry point (absolute or relative path to the // current working directory) , index : './index.js' // Generated packed file extension , extension : '.js' // Do not automaticaly run main entry point (the index) when the DOM is ready // (you have to do require('/') by your self when the dom is ready) , norun : false // Optimise source with uglify , uglify : false // Uglify option : do not mangle names , nomangle : false // Uglify option : do not do deep source optimisation , nosqueeze : false // Uglify option : generate readable source (comments are // allways removed with uglify) , beautify : false // Force re-generation even if a generated file allready exist with // the same hash , force : false // Prepend all the generated file with this string (maybe a copyright) , prepend : null // core & node_modules remplacements // // Node.js core & node_modules remplacement are resolved using the following process : // // - Check for an alternative in options.alternative : { 'package-name': 'alternative-package-name'} // - Check for a browserify alternative : {'http': 'node_modules/http-browserify'} // - Check for a nway alternative : {'http': 'node_modules/http-nway'} // , alternative : {} // Replace a file path resolved by another file path path , substitute : {} // Alias list : // // keys = module alias // value = module resolvable filepath // // nway provide a default alias for main module (entry point) : '/' // // You may define alias to manualy doing a require('myAlias') a module // in the browser. Remember nway hide the real module path in generated sources // as long as you not explicitly provide an alias to them. , alias : {} // Used to force re-generation of packet files when nway version // has change (as the options are used to generate global uniq id) , version : require('../package.json').version // To change global uniq id , catkeyboard : '' // Array of patterns of filepath to exclude form parsing // Use it to enhance compilation speed on allready bundled source without // any commonjs mechanism inside. // // Allowed values are : // - string : Used as a minimatch pattern on absolute file path // - regexp : Used to test absolute file path // - function : Receive the absolute file path and the Module object. Returns a boolean. , noparse : [] // Exclude some dependency from the generation process // You will have to add those dependencies by your self using require.define() // Exclude is an array of minimatch (https://github.com/isaacs/minimatch) wildcards , excludes : [] // compress is a compression function. // default is nway.defaultCompressor (based on uglify) , compress : null // prewrite is a function to execute on the source before // write to disk : the function receive a source, and an object // The object may be (instanceof) : an nway/lib/Bootstrap or a nway/lib/DepNode (packet) // This function MUST always return a source , prewrite : null // Use the onepack builder : all the modules bundled in one source (ignore async splitter) , onepack : false // Parsers : // an hash of extra parser objects : // // - match : minimatch pattern or regex on module filepath or function (with the same // arguments passed to the parse function listed below) // - parse : Parser function that return the parsed (and transformed) source // (see nway/lib/parsers/*) for exemples // // The parse function used in main nway parser is the one with a `match` // pattern that suceed on the filepath // // Each `parse` function receive those arguments : // // - src (string) : Source to parse and transform // - module (object) : Module object created by the main parser function // - makeModule (function): Main parser module creator : receives an absolute // filepath to parse, and return a new Module object. // - options (object): An nway options object // // And return a parsed (and some time transformed) source // // nway defaults parsers (in nway/lib/parsers) : // // - commonjs javascript source (.js) // - json source (comments allowed) (.json) // , parsers : []}
indexResolve.js
lib/indexResolve.jsAPI Summary
- indexResolve()
Try to resolve the index argument :
indexResolve(indexFilePath) public
Try to resolve the index argument :
- a file relative to the current working directory or the main module
- a package name relative to working directory or the main module
function indexResolve(indexFilePath) { var next, result = null; var search = [ // Is a file path relative to current working directory : function() { result = resolve(fs.realpathSync(indexFilePath), process.cwd()); } // Is a file path relative to main module : , function() { result = resolve(fs.realpathSync(indexFilePath), dirname(require.main.filename)); } // Is a package in node_module relative to current working directory : , function() { result = resolve(indexFilePath, process.cwd()); } // Is a package in node_module relative to main module : , function() { result = resolve(indexFilePath, dirname(require.main.filename)); } ] while(next = search.shift()) { try { next(); return result; } catch(e) {} }}
middleware.js
lib/middleware.jsAPI Summary
- middleware()
Middleware contructor
middleware(options) public
Middleware contructor
Return a connectjs middleware for nway.
This middleware force life-cache for the packet files and prevent cache on
the bootstrap file.
The options object
Exemple :
var connect = require('connect') , http = require('http');// The nway options consumed by nway.middleware()// must be the sames of the one used for the compilation// (or at least the 'bootstrap', 'client' and 'extension' keys)var options = {client:'/generated', bootstrap: 'bootstrap.nocache.js', extension: '.js'};var app = connect() .use(require('nway').middleware(options)) .use(connect.static('public')) // Path to public files .listen(3000);http.createServer(app).listen(3000);
function middleware(options) { var options = _.extend({}, require('./defaultOptions.js'), options || {}) , bootstrapPath = join(options.client, options.bootstrap) , regClient = new RegExp('^' + options.client + '\/[^\.\/]*\\'+ options.extension + ') ; return function(req, res, next) { if ('GET' != req.method && 'HEAD' != req.method) return next(); var path = url.parse(req.url).pathname , dir = dirname(path) ; if(dir != options.client) return next(); res.setHeader('X-nway-Middleware', require('../package.json').version); if(path == bootstrapPath) { // No cache res.setHeader('Cache-Control', 'no-cache, must-revalidate, max-age=-1'); res.setHeader('X-nway-Middleware-nocache', '1'); } else if(regClient.test(path)) { // One year cache : res.setHeader('Cache-Control', 'public, max-age=' + cacheMaxAge); res.setHeader('X-nway-Middleware-nocache', '0'); } next(); }}
nway.js
lib/nway.jsMain access to nway builder and nway utilities (returned by require('nway')
) :
- nway() : The builder function
- nway.parser() : The parser
- nway.buildTree() : The tree builder
- nway.middleware() : The nway middleware
// IMPORTvar parser = require('./parser') , buildTree = require('./buildTree') , builder = require('./builder') , middleware = require('./middleware') , defaultCompressor = require('./defaultCompressor');// EXPORT// The main exported function is the builderexports = module.exports = builder;// Expose nway versionexports.version = require('../package.json').version;// Expose some nway utilitiesexports.resolve = require('resolve');exports.parser = parser;exports.buildTree = buildTree;exports.middleware = middleware;exports.defaultCompressor = defaultCompressor;
parser.js
lib/parser.jsAPI Summary
- parser()
Dependency parser
parser(indexFilePath) public
Dependency parser
Parse a file and build dependency module list
TODO : Refactoring, cleanup and documentation
function parser(indexFilePath, options, done) { gdebug('Start'); var modules = [] , deep = 0 , options = _.extend({}, defaults, options) , noparse = options.noparse || [] , parsers = []; ; // try to resolve index filepath : indexFilePath = indexResolve(indexFilePath); if(!indexFilePath) { return done(new Error('Index not found for :' + indexFilePath)); } // Prepare path subsitution by resolving all paths // like we resolve the index : options.substitute = (function() { var substitute = {}; _.each(options.substitute, function(to, from) { debug('substitution prepare: %s to %s', from, to) var _to = indexResolve(to); if(!_to) { debug('Module subsitution error this module can not be resolved (to):' + to) return done(new Error('Module subsitution error this module can not be resolved (to):' + to)); } var _from = indexResolve(from); if(!_from) { debug('Module subsitution error this module can not be resolved (from):' + from) return done(new Error('Module subsitution error this module can not be resolved (from):' + from)); } debug(' substitution result: %s to %s', _from, _to) substitute[_from] = _to; }) return substitute; })(); // Prepare the parser list, option parsers before default parsers : _.each(options.parsers, function(def) { parsers.push(def); }); parsers.push({match: /\.coffee$/, parse: require('./parsers/coffee')}); parsers.push({match: /\.css$/, parse: require('./parsers/css')}); parsers.push({match: /\.html$/, parse: require('./parsers/html')}); parsers.push({match: /\.jade$/, parse: require('./parsers/jade')}); parsers.push({match: /\.json$/, parse: require('./parsers/json')}); parsers.push({match: /\.sass$/, parse: require('./parsers/sass')}); parsers.push({match: /\.stylus$/, parse: require('./parsers/stylus')}); parsers.push({match: /\.js$/, parse: require('./parsers/javascript')}); // Parse a source an return a valid // javascript source injectable in an nway // define() template. // This method call the first valid parser // defined by user (options.parsers) or in // the default parsers function parse(src, module, makeModule, options) { // Find the first valid parser : debug('Find a parser for :', module.path) var result = _.find(parsers, function(def) { // debug(' Check parser match :', def.match) if(def.match instanceof RegExp) { return def.match.test(module.path); } else if ('string' == typeof def.match) { return minimatch(module.path, def.match); } else if ('function' == typeof def.match) { return def.match(src, module, makeModule, options); } else { return false; } }); if(!result) { debug('Warning: No valid parser found for %s. Json string is exported', module.path) return 'module.exports = ' + JSON.stringify(src); } else { debug('Ok parser match :', result.match); return result.parse(src, module, makeModule, options); } } // Create a module object for the index // receives an absolute filepath to parse, and return // a new Module object function makeModule(index) { var stored, src, srcPath, relpath; // Prepare module var module = new Module({ uid: index ,req: [] ,path: index ,relpath: null ,isNodeModule: false ,isAsyncModule: arequirePath === index ,arequirePath: arequirePath ,index: index }); if(~module.path.indexOf('/node_modules/')) module.isNodeModule = true; relpath = path.relative(process.cwd(), module.path); relpath = !/^\./.test(relpath) ? './' + relpath : relpath; module.relpath = relpath; debug('makeModule for %s', relpath); // Check for stored version of this path stored = _.find(modules, function(p) { return module.path === p.path; }); if(stored) { debug('allready stored : return (absolute path match)'); return stored; } // Read source src = fs.readFileSync(module.path, 'utf8'); module.mtime = fs.statSync(module.path).mtime; module.uid = getUID({src: src, path:module.path, options: options}); // Check for stored version with same uid stored = _.find(modules, function(p) { return module.uid === p.uid; }); if(stored) { debug('allready stored : return (md5 hash match)'); return stored; } // Save module modules.push(module); // Do not parse ? module.notparsed = _.find(noparse, function(np) { if(np instanceof RegExp) { return np.test(module.path); } else if ('string' == typeof np) { return minimatch(module.path, np); } else if ('function' == typeof np) { return np(module.path, module); } else { return false; } }); // Parse if(module.notparsed) { debug(' no parse'); } else { debug(' run parse'); src = parse(src, module, makeModule, options); } // Store the module source : module.source = src; // Return the module object debug('end %s', relpath); return module; } var root = makeModule(indexFilePath); process.nextTick(function() { gdebug('End'); done(null, modules); })}
resolve.js
lib/resolve.jsAPI Summary
public
SOURCE// (The MIT License)//// Copyright (c) 2012-2013 Novadiscovery <osproject@novadiscovery.com>//// Permission is hereby granted, free of charge, to any person obtaining// a copy of this software and associated documentation files (the// 'Software'), to deal in the Software without restriction, including// without limitation the rights to use, copy, modify, merge, publish,// distribute, sublicense, and/or sell copies of the Software, and to// permit persons to whom the Software is furnished to do so, subject to// the following conditions://// The above copyright notice and this permission notice shall be// included in all copies or substantial portions of the Software.//// THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,// EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF// MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.// IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY// CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,// TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE// SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.// IMPORTSvar resolve = require('resolve') , join = require('path').join , pathSep = require('path').sep , builtinsPath = join(__dirname, './builtins') , normalize = require('path').normalize , _ = require('underscore') , debug = require('debug')('nway:resolve');module.exports = function(path, basedir, options) { debug(path); var options = options || {}; // Normalize path and prepare path alternatives var isRelative = path.charAt(0) === '.'; var path = normalize(path); // Keep ./ (or .\) for relative path if(isRelative && path.charAt(0) !== '.') { path = '.' + pathSep + path; } // Path parts to build alternative paths var parts = path.split(pathSep); var isAbsolute = path[0] === '' || ~path[0].indexOf(':'); function getPath(alternative) { var aparts = _.clone(parts); if(!isRelative && !isAbsolute && alternative) { aparts[0] = alternative; } return aparts.join(pathSep); } // Resolver search suite: called in series var search = [ // Use standards resolver (relative, absolute or package path) function() { if(resolve.isCore(path)) throw new Error(); debug(' - standard'); return resolve.sync(getPath(options.alternative && options.alternative[parts[0]]), { basedir: basedir }); } // Try to use a browserify package ,function() { if(isRelative || isAbsolute) throw new Error(); debug(' - browserify'); return resolve.sync(getPath(parts[0] + '-browserify'), { basedir: basedir }); } // Try to use a nway package ,function() { if(isRelative || isAbsolute) throw new Error(); debug(' - nway'); return resolve.sync(getPath(parts[0] + '-nway'), { basedir: basedir }); } // Try to use a builtin package ,function() { if(isRelative || isAbsolute) throw new Error(); debug(' - builtin'); return resolve.sync(builtinsPath + part[0], { basedir: basedir }); } ] while(next = search.shift()) { try { return next(); } catch(e) {} } throw new Error("Cannot find module '" + path + "'");}
template.js
lib/templates.jsSingletion template accessor
utils.js
lib/utils.jsAPI Summary
- exports.bytesToSize()
Convert file size to human readable file size
- exports.uid()
Create a short hash string
exports.bytesToSize(bytes) private
Convert file size to human readable file size
exports.bytesToSize = function (bytes) { var sizes = ['B', 'K', 'M', 'G', 'T']; if (bytes == 0) return 'n/a'; var i = parseInt(Math.floor(Math.log(bytes) / Math.log(1024))); return Math.round(bytes / Math.pow(1024, i), 2) + ' ' + sizes[i];}
exports.uid(str) public
Create a short hash string
exports.uid = function(str) { if('object' === typeof str) { str = JSON.stringify(str, function(k, v) { if('function' === typeof v) return v.toString(); return v; }) } return crc.hex32(crc.crc32(str));}
onepack.js
lib/builders/onepack.jsAPI Summary
- onepack()
The onepack builder : generate a standalone file with all
the modules packed in.
onepack(options, callback) public
The onepack builder : generate a standalone file with all
the modules packed in.
The result object returned to the builder is the generated source
function onepack(options, callback) { total('Start'); // Total is a time debug() var options = _.extend({}, defaults, options) , callback = (callback || function() {}) , modules , generated ; debug('prepare'); parser(options.index, options, function(err, modules) { debug('Parser end, get all modules contents.'); var allmodules = _.invoke(modules, 'generate').join('\n'); debug('Build the tree ...'); var tree = buildTree(options, modules, true); debug('Prepare datas ...'); var data = { allmodules: allmodules ,loader: templates.loaderonepack } var bootstrap = new Bootstrap(); generated = bootstrap.generate(tree, options, templates.bootstrap, data ); callback(null, generated); });}
standard.js
lib/builders/standard.jsAPI Summary
- NoErrorEnd()
No error constructor
- standard()
The standard builder : generate the bootstrap and the packets files
in the output folder. - initialize()
Step 1
- createDirectories()
Step 2
- excludeBundles()
Step 3
- writeBundles()
Step 5
- writeBootstrap()
Step 8
NoErrorEnd() private
No error constructor
NoErrorEnd is used by the asynchronous process
to stop the processus without returning an error to
the global callback. This is a kind of break when
a destination packet file is allready generated.
function NoErrorEnd() {}
standard(options, callback) public
The standard builder : generate the bootstrap and the packets files
in the output folder.
The result object returned to the builder is :
{ // The dependencie tree (generated by buildTree) , tree : tree // All the packets objects , packets : packets // List of the packet that have been generated // (some packets are not generated if the destination file // allready exists) , incBundles : incBundles}
function standard(options, callback) { total('Start'); // Total is a time debug() var options = _.extend({}, defaults, options) , callback = (callback || function() {}) , tree , packets , incBundles = [] ; // BUILD FLOW
initialize() private
Step 1
Prepare builder information using nway.prepare() :
modules object, root module, sources, tree and packet list
function initialize(done) { debug('prepare'); parser(options.index, options, function(err, modules) { if(err) return done(err) debug('Parser return, now build the tree...'); tree = buildTree(options, modules, true); debug('Tree builded'); // Extract from tree as a flatten list of packets to compile packets = tree.getAllBundles(options); done(null); }) }
createDirectories() private
Step 2
Create the required directories :
- output directory
function createDirectories(done) { debug('createDirectories()'); var paths = [ options.output ]; function iterator(dirpath, next) { mkdirp(dirpath, '0755', next); }; async.forEach(paths, iterator, done); }
excludeBundles() private
Step 3
Apply packet the packet exclusion process : if a packet is allready generated
(with the same hash key) then the packet is removed from the generator
stack.
If the generator stack is empty, then the NoErrorEnd() object returned to the
callback stop the generation process.
function excludeBundles(done) { debug('excludeBundles()'); // With force options to true : do not exclude any packet if(options.force) { incBundles = packets; return done(); } // Check the existence of all the packets output files : async.forEach(packets, function(packet, next) { fs.stat(packet.makePath(options), function(err) { // Packet doesn't exits : add it to packets to compile : if(err) incBundles.push(packet); next(); }); }, function() { if(!incBundles.length) { // Stop generation process if all the packets are allready // generated //return done(new NoErrorEnd()); // !! Bug : we nee to rewrite the //bootstrap anyway !! } done(); }); }
writeBundles() private
Step 5
Call write() on all packets
function writeBundles(done) { debug('writeBundles()'); async.forEachSeries(incBundles, function(packet, next) { packet.write(options, next); }, done); }
writeBootstrap() private
Step 8
Wite the boostrap file
function writeBootstrap(done) { var bootstrap = new Bootstrap(); bootstrap.write(tree, options, done); } // Run asynchronous flow : async.series([ initialize // Step 1 , createDirectories // Step 2 , excludeBundles // Step 3 , writeBundles // Step 5 , writeBootstrap // Step 8 ],function(err) { total('End'); // Prepare the result object returned to the // builder() callback var result = { packets : packets , incBundles : incBundles , tree : tree } if(!err || (err instanceof NoErrorEnd)) { return callback(null, result); } else { return callback(err, result); } });}
Demo 01: simple.demo.js
demo/01_simple/simple.demo.jsGo into the demo/01_simple folder:
cd demo/01_simple
Execute the demo application using node to see what it do:
node src/index.js
Now execute the simple.demo.js
to generate the client version
of this application, then open the public/index.html in
your browser. In the javascript console, the output is the same
of the execution above :
node simple.demo.js
open public/index.html
The both output those messages:
index.js: I require foo.js
foo.js: I require bar.js
bar.js: I export a function
foo.js: I add a function run() to my export object
index.js: I call run() on foo
foo.js: In my run(). Now I call bar.
bar.js: I say hello to Foo
The code of simple.demo.js
use nway API and the result
is the same with the nway command below :
nway src/index.js
Below, the simple.demo.js
source :
var nway = require('../..'); // = require('nway')nway({ index: __dirname + '/src/index.js'});
Demo 02: arequire.demo.js
demo/02_arequire/arequire.demo.jsGo into the demo/02_arequire folder:
cd demo/02_arequire
Execute the demo application using node to see what it do:
node src
Now execute the arequire.demo.js
to generate the client version
of this application, then open the public/index.html in
your browser. In the javascript console, the output is the same
of the execution above :
node arequire.demo.js
open public/index.html
The both output those messages:
index.js: I get a arequire() function for my module
index.js: For demo purpose, I export my application object
index.js: Go in the FOO part of the application:
index.js: goFoo(). Asynchronously require foo.js. This is an application split point.
foo.js: I am a huge part of the application and I export a function
index.js: foo.js is imported. Execute it exported function foo().
foo.js: I am executed
index.js: Now go in the BAR part of the application:
index.js: goBar(). Asynchronously require bar.js. This is an application split point.
bar.js: I am a huge part of the application and I export a function
index.js: bar.js is imported. Execute it exported function bar().
bar.js: I am executed
The code of arequire.demo.js
use nway API and the result
is the same with the nway command below :
nway src
Below, the arequire.demo.js
source :
var nway = require('../..'); // = require('nway')nway({ index: __dirname + '/src/index.js'});
Demo 03: substitute.demo.js
demo/03_substitute/substitute.demo.jsGo into the demo/03_substitute folder:
cd demo/03_substitute
Execute the demo application using node to see what it do:
node src
Now execute the substitute.demo.js
to generate the client version
of this application, then open the public/index.html in
your browser. In the javascript console, the output is intentionaly
NOT THE SAME of the node execution :
node arequire.demo.js
open public/index.html
The node src
result is :
index.js: Require and execute foobar.js
foobar.js: Hello
The browser result is :
index.js: Require and execute foobar.js
And in the page you see : "foobar.js: Hello I'am the browser alternative !"
BUT remember that you do not have to do this kind of thing for testing
as some awesome lib offert headless browser for testing browser applications
like Zombie.js, PhantomJS
or CasperJS
Below, the substitute.demo.js
source :
var nway = require('../..'); // = require('nway')nway({ index: __dirname + '/src/index.js' , substitute: { // Substitute the src/foobar.js by is the browser-only version : "./src/foobar.js" : "./src/foobar.browser.js" }});
Demo 04: jade.demo.js
demo/04_jade/jade.demo.jsGo into the demo/04_jade folder:
cd demo/04_jade
Execute the demo application using node to see what it do:
node src/index.js
Now execute the jade.demo.js
to generate the client version
of this application, then open the public/index.html in
your browser.
node jade.demo.js
open public/index.html
The code of jade.demo.js
use nway API and the result
is the same with the nway command below :
nway src
Below, the jade.demo.js
source :
var nway = require('../..'); // = require('nway')nway({ index: __dirname + '/src'});
Demo 05: coffee.demo.js
demo/05_coffee/coffee.demo.jsGo into the demo/05_coffee folder:
cd demo/05_coffee
Execute the demo application using node to see what it do:
node src
Now execute the coffee.demo.js
to generate the browser version
of this application, then open the public/index.html in
your browser, and see what happen in the console.
node coffee.demo.js
open public/index.html
The code of coffee.demo.js
use nway API and the result
is the same with the nway command below :
nway src
Below, the coffee.demo.js
source :
var nway = require('../..'); // = require('nway')nway({ index: __dirname + '/src'});
Bootstrap(init) public
Bootstrap constructor