../Libraries/javascript/ in runtime

Using Javascript from SpiderBasic
User avatar
useful
Posts: 116
Joined: Tue Feb 25, 2014 1:15 pm

../Libraries/javascript/ in runtime

Post by useful »

I packed the js and css with

https://github.com/terser/html-minifier-terser
https://github.com/terser/terser

In doing so, I found that cbtreeIcons.css is zero length.
I'm sure that not all files are needed in runtime.
How can I figure out the composition and minimize the number of files in ../Libraries/javascript/ to what's needed at runtime only?

p.s. And how to go further, i.e. to minimize even more, to what is used only in a particular application?
p.p.s. I put the application on the resource with payment for the amount of storage and traffic on actual consumption. Therefore, size does matter.
2B or not 2B = FF
User avatar
Peter
Posts: 1086
Joined: Mon Feb 24, 2014 10:17 pm
Location: 127.0.0.1:9080
Contact:

Re: ../Libraries/javascript/ in runtime

Post by Peter »

useful wrote: Sat Feb 25, 2023 6:41 am I packed the js and css with

https://github.com/terser/html-minifier-terser
https://github.com/terser/terser
viewtopic.php?t=1893 ;)

useful wrote: Sat Feb 25, 2023 6:41 amI'm sure that not all files are needed in runtime.
Yes, there is still room for optimization. For example, the paper library with a size of 223 KB is always included, although it is only used for the vector commands: viewtopic.php?t=1258
User avatar
useful
Posts: 116
Joined: Tue Feb 25, 2014 1:15 pm

Re: ../Libraries/javascript/ in runtime

Post by useful »

I'm pretty sure that by analyzing the resulting app.html and app.js from the create web app you can figure out which files are needed and which are not.
But I don't have any experience with that.
Only old experience in another tool (shroud) :roll:
http://www.rapideuphoria.com/bind.htm

And I'd like a similar.
2B or not 2B = FF
User avatar
useful
Posts: 116
Joined: Tue Feb 25, 2014 1:15 pm

Re: ../Libraries/javascript/ in runtime

Post by useful »

!!!
sbmongoose.exe
mongoose.conf ->
access_log_file d:\websrv\l.txt
And in the protocol we see everything that the browser asks the server.
!!!
It was so simple it was even a little funny. :D :D :D

p.s. Or in the server protocol that you use for your application

p.p.s. It remains to dig deep into libxxx.js. There's probably a lot of unnecessary stuff in there, too. 8-)
2B or not 2B = FF
User avatar
useful
Posts: 116
Joined: Tue Feb 25, 2014 1:15 pm

Re: ../Libraries/javascript/ in runtime

Post by useful »

For example, GadgetOverview.sb (go.html) requires only this

/go.html
/go.js
/favicon.png
/Data/ToolBar/New.png
/Data/Spider.png

/lib240/platform.js
/lib240/init.js
/lib240/main.js
/lib240/dojo/dojo.js
/lib240/dojo/themes/flat/flat.css
/lib240/dojo/dgrid/css/dgrid.css
/lib240/themes/flat/window.css
/lib240/xdate.min.js
/lib240/dojo/nls/dojo_(Browser locale).js
/lib240/put.min.js
/lib240/dojo/cbtree/errors/CBTErrors.json
/lib240/jquery.min.js
/lib240/filesaver.min.js
/lib240/canvas-toBlob.min.js
/lib240/interact.min.js
/lib240/mousetrap.min.js
/lib240/paper-full.min.js
/lib240/media/background.png
/lib240/dojo/resources/blank.gif
/lib240/jquery.injectCSS.js
/lib240/jquery-ui.custom.min.js
/lib240/jquery.blockUI.min.js
/lib240/mousetrap-global-bind.min.js
/lib240/dojo/dgrid/css/images/ui-icons_222222_256x240.png
/lib240/dojo/themes/flat/font-awesome-4.5.0/fonts/fontawesome-webfont.woff2
/lib240/dojo/themes/flat/images/progressBarStrips.png

p.s. And there's definitely something on this list that isn't really needed.
2B or not 2B = FF
the.weavster
Posts: 220
Joined: Sat Mar 01, 2014 3:02 pm

Re: ../Libraries/javascript/ in runtime

Post by the.weavster »

If you use the following (not well tested ;) ) Python 3 script to serve your SB application it will copy any file it serves into a subfolder named "dist".
Just save a copy in your SB project folder and run it from there.

Code: Select all

#!/usr/bin/env python
# coding: utf-8
import mimetypes, os, shutil, socket
from http.server import ThreadingHTTPServer, BaseHTTPRequestHandler

ROOT_DIR = os.path.abspath(os.path.dirname(__file__))
DIST_DIR = os.path.join(ROOT_DIR, 'dist')
if not os.path.exists(DIST_DIR): os.makedirs(DIST_DIR)

def main():
    PORT = 8081
    server = ThreadingHTTPServer(('', PORT), Handler)
    print('SpiderServer (use <Ctrl-C> to stop)')
    print('Listening at: http://{}:{}'.format(socket.gethostbyname(socket.gethostname()), PORT))
    server.serve_forever()
    
    
class Handler(BaseHTTPRequestHandler):
    
    # override some built-ins as boring stuff
    # is output to the terminal ...
    def finish(self, *args, **kw):
      try:
        if not self.wfile.closed:
          self.wfile.flush()
          self.wfile.close()
      except socket.error:
        pass
      self.rfile.close()
            
    def handle(self):
        try:
            BaseHTTPRequestHandler.handle(self)
        except socket.error:
            pass
    
    def log_error(self, fmt, *args):
        return
    
    def log_message(self, fmt, *args): 
        return
    
    # this adds a served file to the 'dist' folder ...    
    def dist_add(self, fpf, fpt):
        os.makedirs(os.path.dirname(fpt), exist_ok=True)
        shutil.copy(fpf, fpt)
        
    def do_GET(self):
        if self.path == "/": self.path = "/index.html"
        s   = self.path.split("/")
        fpe = os.sep.join(s)
        pos = fpe.find('?')
        if pos != -1: # chop any parameters off
            fpe = fpe[0:pos]
        fpf = ROOT_DIR + fpe
        if os.path.isfile(fpf):
            sz = os.path.getsize(fpf)
            mt = mimetypes.guess_type(fpf)
            f  = open(fpf, 'rb')
            fc = bytes(f.read())
            f.close()
            self.send_reply(fc, sz, 200, mt[0])
            
            # copy served file ...
            fpd = DIST_DIR + fpe
            self.dist_add(fpf, fpd)
        else:
            msg = bytes('<html><body>File not found</body></html>', 'utf-8')
            szr = len(msg)
            self.send_reply(msg, szr, 404, 'text/html')
            
    def send_reply(self, obj, szr, response_code, content_type):
        self.send_response(response_code)
        self.send_header('content-type', content_type)
        self.send_header('content-length', szr)
        self.end_headers()
        self.wfile.write(obj) 


if __name__ == '__main__':
    main()
User avatar
useful
Posts: 116
Joined: Tue Feb 25, 2014 1:15 pm

Re: ../Libraries/javascript/ in runtime

Post by useful »

viewtopic.php?p=9118#p9118

I don't see the point of connecting python as well.
A simple analysis of the protocol (text file) of the server gives a list as it is.
For now it is more important that this list is clearly redundant.
2B or not 2B = FF
the.weavster
Posts: 220
Joined: Sat Mar 01, 2014 3:02 pm

Re: ../Libraries/javascript/ in runtime

Post by the.weavster »

useful wrote: Fri Mar 03, 2023 12:20 pm I don't see the point of connecting python as well.
The Python script doesn't just give you a list, it copies the required files into a "dist" folder.
the.weavster
Posts: 220
Joined: Sat Mar 01, 2014 3:02 pm

Re: ../Libraries/javascript/ in runtime

Post by the.weavster »

Here's an updated version which adds compression:

Code: Select all

#!/usr/bin/env python
# coding: utf-8
import json, mimetypes, os, shutil, socket, urllib.parse, urllib.request
from http.server import ThreadingHTTPServer, BaseHTTPRequestHandler

DIST_COMPRESS = True
ROOT_DIR = os.path.abspath(os.path.dirname(__file__))
DIST_DIR = os.path.join(ROOT_DIR, 'dist')
if not os.path.exists(DIST_DIR): os.makedirs(DIST_DIR)

def main():
    PORT = 8081
    server = ThreadingHTTPServer(('', PORT), Handler)
    print('SpiderServer (use <Ctrl-C> to stop)')
    print('Listening at: http://{}:{}'.format(socket.gethostbyname(socket.gethostname()), PORT))
    server.serve_forever()
    
    
class Handler(BaseHTTPRequestHandler):
    
    # override some built-ins as boring stuff
    # is output to the terminal ...
    def finish(self, *args, **kw):
      try:
        if not self.wfile.closed:
          self.wfile.flush()
          self.wfile.close()
      except socket.error:
        pass
      self.rfile.close()
            
    def handle(self):
        try:
            BaseHTTPRequestHandler.handle(self)
        except socket.error:
            pass
    
    def log_error(self, fmt, *args):
        return
    
    def log_message(self, fmt, *args): 
        return
    
    # this adds a served file to the 'dist' folder ...
    def dist_add(self, fpf, fpt, ftype=''):
        os.makedirs(os.path.dirname(fpt), exist_ok=True)
        shutil.copy(fpf, fpt)
        if DIST_COMPRESS and ftype in ['text/css', 'text/html', 'text/javascript']:
            self.dist_compress(fpt, ftype)
    
    # this minifies html/css/js ...    
    def dist_compress(self, fp, ftype):
        compressor_url = "https://htmlcompressor.com/compress"
        req_data = { 'code_type': 'html' }
        if ftype == 'text/css':
            req_data['code_type'] = 'css'
        elif ftype == 'text/javascript':
            req_data['code_type'] = 'js'
            req_data['js_engine'] = 'closure'
        
        size_before = os.path.getsize(fp)
        with open(fp, "r+") as f:
            req_data['code'] = f.read()
            enc_data = bytes(urllib.parse.urlencode(req_data), 'utf-8')
            req = urllib.request.Request(
                url    = compressor_url, 
                data   = enc_data, 
                method = 'POST')
            with urllib.request.urlopen(req) as response:
                if response.status == 200:
                    f.seek(0)
                    f.write(response.read().decode('utf-8'))
                    f.truncate()
                    size_after = os.path.getsize(fp)
                    print(fp + " compressed from " + str(size_before) + " to " + str(size_after))
        
    def do_GET(self):
        if self.path == "/": self.path = "/index.html"
        s   = self.path.split("/")
        fpe = os.sep.join(s)
        pos = fpe.find('?')
        if pos != -1: # chop any parameters off
            fpe = fpe[0:pos]
        fpf = ROOT_DIR + fpe
        if os.path.isfile(fpf):
            sz = os.path.getsize(fpf)
            mt = mimetypes.guess_type(fpf)
            f  = open(fpf, 'rb')
            fc = bytes(f.read())
            f.close()
            self.send_reply(fc, sz, 200, mt[0])
            
            # copy served file ...
            fpd = DIST_DIR + fpe
            self.dist_add(fpf, fpd, mt[0])
        else:
            msg = bytes('<html><body>File not found</body></html>', 'utf-8')
            szr = len(msg)
            self.send_reply(msg, szr, 404, 'text/html')
            
    def send_reply(self, obj, szr, response_code, content_type):
        self.send_response(response_code)
        self.send_header('content-type', content_type)
        self.send_header('content-length', szr)
        self.end_headers()
        self.wfile.write(obj) 


if __name__ == '__main__':
    main()
Post Reply