Thursday, 26 November 2020

Client Side export csv or xlsx

 CSV or XSLX export on client requires a lot of memory, it must uses BLOB(Binary large object) to avoid browser crash.


Blob means “Binary Large Object” and it’s an opaque representation of a chunk of bytes. Web Browsers implement a Blob object, which is responsible for holding data.

https://medium.com/javascript-in-plain-english/javascript-blob-why-is-it-useful-20c372dfca00


BLOB size and implementation  is different via browser :

CHROME :

Blobs are created in a renderer process, where their data is temporarily held for the browser (while Javascript execution can continue). When the browser has enough memory quota for the blob, it requests the data from the renderer. All blob data is transported from the renderer to the browser. Once complete, any pending reads for the blob are allowed to complete. Blobs can be huge (GBs), so quota is necessary.

If the in-memory space for blobs is getting full, or a new blob is too large to be in-memory, then the blob system uses the disk. This can either be paging old blobs to disk, or saving the new too-large blob straight to disk.

https://chromium.googlesource.com/chromium/src/+/master/storage/browser/blob/README.md#:~:text=Blobs%20are%20created%20in%20a,the%20renderer%20to%20the%20browser.



Export csv using JS and BLOB:

From array of arrays input [ [header1, header2,header3], [data1,data2,data3]] (Rows)

exportToCsv(filename: string, rows) {

    if (!rows || !rows.length) {

        return;

    }


    const separator = ',';

    const keys = Object.keys(rows[0]);


    const csvContent =

        rows.map(row => {

            return keys.map(k => {

                let cell = row[k] === null || row[k] === undefined ? '' : row[k];

                cell = cell instanceof Date

                    ? cell.toLocaleString()

                    : cell.toString().replace(/"/g, '""');

                if (cell.search(/("|,|\n)/g) >= 0) {

                    cell = `"${cell}"`;

                }

                return cell;

            }).join(separator);

        }).join('\n');

    const blob = new Blob([csvContent], { type: 'text/csv;charset=utf-8;' });


    if (navigator.msSaveBlob) { // IE 10+

        navigator.msSaveBlob(blob, filename);

    } else {

        const link = document.createElement('a');

        if (link.download !== undefined) {

            // Browsers that support HTML5 download attribute

            const url = URL.createObjectURL(blob);

            link.setAttribute('href', url);

            link.setAttribute('download', filename);

            link.style.visibility = 'hidden';

            document.body.appendChild(link);

            link.click();

            document.body.removeChild(link);

        }

    }

}


The join() method returns the array as a string.

The elements will be separated by a specified separator. The default separator is comma (,).



Export array to xlsx is more diffcult(convert string to utf-16 binary needed), can be done using sheetjs plugin as done below:

https://redstapler.co/sheetjs-tutorial-create-xlsx/

https://github.com/SheetJS/sheetjs/blob/master/xlsx.js


Creating a Workbook

Now let’s start with creating a new workbook by calling book_new() utility function which will return an empty workbook object.

var wb = XLSX.utils.book_new();

You can update the workbook properties such as title, subject, author with wb.Props.


wb.Props = {
Title: "SheetJS Tutorial",
Subject: "Test",
Author: "Red Stapler",
CreatedDate: new Date(2017,12,19)
};

Now we have the workbook, the next step is to create a worksheet and add it to the workbook. First, you’ll need to assign a new sheet name and push it to the SheetNames array.

wb.SheetNames.push("Test Sheet");

Then, for the content inside the sheet, you have several options. You have create a sheet from array of array, JSON or html table. For this tutorial, I’m going to use array of array. The structure is quite straightforward. Each array represent the row data and the members are the cell content.

var ws_data = [['hello' , 'world']]; //a row with 2 columns

Now create the sheet from this array by using aoa_to_sheet()

var ws = XLSX.utils.aoa_to_sheet(ws_data);

And assign the sheet object to the workbook Sheets array.

wb.Sheets["Test Sheet"] = ws;

Congratulation, now you have created a workbook and a worksheet with first row of data. The next step is to generate an xlsx file.

Exporting Workbook for Download

We need to export the workbook as xlsx binary. Use write function then pass the bookType as xlsx and output Type as binary

var wbout = XLSX.write(wb, {bookType:'xlsx', type: 'binary'});

We now have our xlsx binary data on wbout var. However, the correct content type for excel file is octet stream so you’ll need to convert the binary data into octet. We can achieve that by using arrayBuffer, UInt8Array and bit operation like this.

function s2ab(s) {
var buf = new ArrayBuffer(s.length); //convert s to arrayBuffer
var view = new Uint8Array(buf); //create uint8array as viewer
for (var i=0; i<s.length; i++) view[i] = s.charCodeAt(i) & 0xFF; //convert to octet
return buf;
}

We’re going to utilize Filesaver.js and Blob to handle the file saving for cross browser support. Use saveAs() function and create a new Blob object from octet array. Set the content type as octet-stream. follow by excel file naming that you would like.

$("#button-a").click(function(){
saveAs(new Blob([s2ab(wbout)],{type:"application/octet-stream"}), 'test.xlsx');
});

We’re ready; Let’s test it!

Tuesday, 24 November 2020

Array Join & Object.keys

Join

Convert the elements of an array into a string: 

var c = [ ['a','b'], ['c','d']]


c.map( row => {return row.join() }).join('\n');

// Map returns ['a,b', 'c,d'];

    // "a,b

c,d"


Object.keys

var a =['a', 'b']

Object.keys(a);

// Return array of keys

["0", "1"]

Monday, 23 November 2020

Power Shell to check MAC of PC

 Get-WmiObject win32_networkadapterconfiguration | select description, macaddress


https://devblogs.microsoft.com/scripting/powertip-use-powershell-to-find-mac-address/

Thursday, 19 November 2020

Angular scss Import and use variable from other scss

 

  • The definition of paths depends on your version of Angular. In our project, old versions use angular-cli.json and new ones use angular.json:

    for "@angular/cli": "~1.7.4" use angular-cli.json:

    "stylePreprocessorOptions": {
        "includePaths": [
          "../src",
          "./scss"
        ]
      },
    

    for "@angular/cli": "~7.0.6":

    "stylePreprocessorOptions": {
        "includePaths": [
           "./src",
           "./src/scss"
        ]


https://stackoverflow.com/questions/42865697/scss-import-relative-to-root


or 

@import "{}/node_modules/module-name/stylesheet";




Difference between utf8 vs base 64, and include base 64 encoded string in HTML

 UTF-8 and UTF-16 are methods to encode Unicode strings to byte sequences.

See: The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!)

Base64 is a method to encode a byte sequence to a string.

So, these are widely different concepts and should not be confused.

Things to keep in mind:

  • Not every byte sequence represents an Unicode string encoded in UTF-8 or UTF-16.

  • Not every Unicode string represents a byte sequence encoded in Base64.



https://stackoverflow.com/questions/3866316/whats-the-difference-between-utf8-utf16-and-base64-in-terms-of-encoding#:~:text=UTF%2D8%20is%20like%20the,Text.


HTML
https://stackoverflow.com/questions/17090571/is-there-a-way-to-set-background-image-as-a-base64-encoded-image

.backgroundA {
    background-image: url("data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAADIAAAAyCAIAAACRXR/mAAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAAyJpVFh0WE1MOmNvbS5hZG9iZS54bXAAAAAAADw/eHBhY2tldCBiZWdpbj0i77u/IiBpZD0iVzVNME1wQ2VoaUh6cmVTek5UY3prYzlkIj8+IDx4OnhtcG1ldGEgeG1sbnM6eD0iYWRvYmU6bnM6bWV0YS8iIHg6eG1wdGs9IkFkb2JlIFhNUCBDb3JlIDUuMC1jMDYwIDYxLjEzNDc3NywgMjAxMC8wMi8xMi0xNzozMjowMCAgICAgICAgIj4gPHJkZjpSREYgeG1sbnM6cmRmPSJodHRwOi8vd3d3LnczLm9yZy8xOTk5LzAyLzIyLXJkZi1zeW50YXgtbnMjIj4gPHJkZjpEZXNjcmlwdGlvbiByZGY6YWJvdXQ9IiIgeG1sbnM6eG1wPSJodHRwOi8vbnMuYWRvYmUuY29tL3hhcC8xLjAvIiB4bWxuczp4bXBNTT0iaHR0cDovL25zLmFkb2JlLmNvbS94YXAvMS4wL21tLyIgeG1sbnM6c3RSZWY9Imh0dHA6Ly9ucy5hZG9iZS5jb20veGFwLzEuMC9zVHlwZS9SZXNvdXJjZVJlZiMiIHhtcDpDcmVhdG9yVG9vbD0iQWRvYmUgUGhvdG9zaG9wIENTNSBNYWNpbnRvc2giIHhtcE1NOkluc3RhbmNlSUQ9InhtcC5paWQ6RDUxRjY0ODgyQTkxMTFFMjk0RkU5NjI5MEVDQTI2QzUiIHhtcE1NOkRvY3VtZW50SUQ9InhtcC5kaWQ6RDUxRjY0ODkyQTkxMTFFMjk0RkU5NjI5MEVDQTI2QzUiPiA8eG1wTU06RGVyaXZlZEZyb20gc3RSZWY6aW5zdGFuY2VJRD0ieG1wLmlpZDpENTFGNjQ4NjJBOTExMUUyOTRGRTk2MjkwRUNBMjZDNSIgc3RSZWY6ZG9jdW1lbnRJRD0ieG1wLmRpZDpENTFGNjQ4NzJBOTExMUUyOTRGRTk2MjkwRUNBMjZDNSIvPiA8L3JkZjpEZXNjcmlwdGlvbj4gPC9yZGY6UkRGPiA8L3g6eG1wbWV0YT4gPD94cGFja2V0IGVuZD0iciI/PuT868wAAABESURBVHja7M4xEQAwDAOxuPw5uwi6ZeigB/CntJ2lkmytznwZFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYW1qsrwABYuwNkimqm3gAAAABJRU5ErkJggg==");
}

Wednesday, 18 November 2020

X509 SSL certificate

 x509 specifies the format of SSL server certificates :

No matter its intended application(s), each X.509 certificate includes a public keydigital signature, and information about both the identity associated with the certificate and its issuing certificate authority (CA):

  • The public key is part of a key pair that also includes a private key. The private key is kept secure, and the public key is included in the certificate. This public/private key pair:
    • Allows the owner of the private key to digitally sign documents; these signatures can be verified by anyone with the corresponding public key.
    • Allows third parties to send messages encrypted with the public key that only the owner of the private key can decrypt.
  • digital signature is an encoded hash (fixed-length digest) of a document that has been encrypted with a private key. When an X.509 certificate is signed by a publicly trusted CA, such as SSL.com, the certificate can be used by a third party to verify the identity of the entity presenting it.
https://www.ssl.com/faqs/what-is-an-x-509-certificate/

HTTPS reset CALL with two servers

 Suppose server A(client) wants to make REST API call via SSL to server B

1) Server B needs to allow CORS(I.E allowing making requests from non origin domain and port)

2) Server A and Server B must complete the process of mutual authentication after TCP handshake.

3) Mutual authentication is Server A needs to present its server SSL to server B, and serverB needs to present its server SSL certificate to server A. Both server validate using open ssl / CURL then if its valid, server A(client) can create a symmetric key and use server B public key to encrypt, and then start communication 

4) One example would be serviceNow making REST API CALL to ansible tower via SSL.

ServviceNow requires an upload of Ansible tower server SSL certificate to cert store, then validate it when making https initial request with ansible tower. If two certs matched, service now then validates the certificate via open ssl. (Ansible tower in this case does not require servicenow certificate). When all valid, data are ecnrypted using symmetric key created from ServiceNow, and ecnrypted using public key from Ansible tower server SSL certificate.

https://docs.servicenow.com/bundle/paris-platform-administration/page/administer/general/task/t_UploadATrustedServerCertificate.html

https://hi.service-now.com/kb_view.do?sysparm_article=KB0696599

https://learn.akamai.com/en-us/webhelp/iot/internet-of-things-over-the-air-user-guide/GUID-21EC6B74-28C8-4CE1-980E-D5EE57AD9653.html#:~:text=Mutual%20authentication%2C%20also%20known%20as,certificates%20to%20prove%20their%20identities.


Mutual Authentication

Mutual authentication, also known as two-way authentication, is a security process in which entities authenticate each other before actual communication occurs. In a network environment, this requires that both the client and the server must provide digital certificates to prove their identities. In a mutual authentication process, a connection can occur only if the client and the server exchange, verify, and trust each other’s certificates. The certificate exchange occurs by means of the Transport Layer Security (TLS) protocol. The core of this process is to make sure that clients communicate with legitimate servers, and servers cooperate only with clients who attempt access for legitimate purposes.

The mutual authentication process involves the following certificates:

Root CA certificate
Used to identify a certificate authority (CA) that signed a client's certificate. It is a self-signed certificate that meets the X.509 standard, defining the format of public key certificates. In IoT products, clients upload a root CA certificate or a certificate chain to verify that the certificates that client devices send to edge servers can be trusted. See Upload the Mutual Authentication root certificate.
Server SSL certificate
Used to identify edge servers to client devices over TLS and to establish a secure connection during the TLS handshake. It is the enhanced TLS certificate that you provide in your property configuration. See Associate a property hostname to an edge hostname.
Client SSL certificate
Used to identify client devices to edge servers over TLS. This certificate must meet the X.509 standard, defining the format of public key certificates.
The process of authenticating and establishing an encrypted channel using certificate-based mutual authentication involves the following steps:


  1. During configuration, administrators provide a root CA certificate or a certificate chain used to sign certificates on client devices. They can do it in Certificate Provisioning System (CPS). See Upload the Mutual Authentication root certificate.
  2. The OTA Updates application deploys the certificate chain to the Akamai Platform.
  3. Once the signing CA certificates propagate across the Akamai Platform, client device can connect by using MQTT, HTTP, or WebSocket protocols and request access to a topic.
  4. The edge server presents its certificate to the client device.
  5. The client device checks its list of trusted CAs and verifies the server’s certificate.
  6. If successful, the client device sends its certificate to the edge server.
  7. The edge server checks its list of CAs and verifies the client device’s certificate.
  8. If successful, a secure connection between the server and the client device is established.