As I said in the article about the benchmarking-- I definitely did it the "quick and dirty" way, testing the whole application 2.9 vs 3.0 on just my device-- not testing specifically for difference in power usage, GPU usage, etc. I would love to have time/resources enough to more microtests as you describe.
Micrio used OffscreenCanvas for a long time. Turns out (apart from occasional flickering screens in Safari) while it did great for Canvas2D operations, it didn't seem to make a difference for WebGL. It actually adds more rendering steps, since you're basically rendering to a framebuffer first.
As someone else pointed out below, with loading the textures using WebWorkers, I indeed didn't use the transferable objects, so it was basically copying a lot of buffers, explaining the performance hits. I'll definitely be experimenting with that.
About the gzipping solution; I've just run a benchmark. It takes 14ms to gunzip & parse the JS base64 to ES6 string, and 59ms to run it inside the `new Function()`-parser. Comparing that to 65% CPU saved per frame drawn (not to mention the general V8 ES6 optimizations compared to the previous ES5 JS-version), I think the cost is worth it :)
I have to admit that I didn't dive 100% into what made this difference exactly. It just worked better without webworkers.
I just dived into the git history, and it turns out I didn't use the buffers as transferable. Perhaps that was it! I'll definitely check that out later, thank you for pointing this out!
No problem. I just happen to be working on a somewhat similar project. It's strictly 360 photos and it's built around our company's foregin language instruction services, but I've had to deal with/am currently dealing with several similar issues. Indeed, the next major tech change I had planned was to also implement my own WebGL code and get rid of Three.js. I'd like to see if I could get the WebGL code running in a WebWorker with OffscreenCanvas, but I'm as-yet unsure if that will work with WebXR.
I wholeheartedly agree with this. V8 is definitely incredible impressive and keeps improving. However Micrio uses a lot of low level Matrix4 maths, which is substantially faster in Wasm.
The second point is also great; having the base Micrio engine as a Wasm module now really makes me itch to use it in different environments. Perhaps I can use it in a serverside rendering flow, or even on an embedded device with a touch screen? Perhaps even as a native mobile app component in the future.
Also from the developer point of view it offers improvements over JS. Type/memory wise it gives more control. It's great to be able to use (u)int, f32 and f64 types, whereas with JS this is impossible from a coding perspective. The buffers passed from Wasm to WebGL are all Float32Arrays and are casted from f64's to f32's manually; giving me that 100% control instead of the black box that V8 gives.
Just for the record, the Wasm implementation in Chrome is a subsystem of V8. It's integrated into the main codebase and shares a lot of runtime services with the JavaScript support, primarily the optimizing compiler tier. So improvements to the (backend) of the optimizing compiler in V8 benefit both JavaScript and Wasm. There is also a baseline JIT compiler which is specifically built solely for faster Wasm startup and not used by JS.
The team (my former team) that maintains Wasm is a subteam of V8 and they all work together. This is unlike (P)NaCl which was both a separate team and a separate subsystem within Chrome. All of the former NaCl people work on Wasm now, primarily tooling and standards.
That's true. But wasm also required some extra codegen and optimizations for i64 types for example (also simd and atomics). In addition, wasm will have more specific CFG optimizations for WebAssembly. For example
https://bugs.chromium.org/p/v8/issues/detail?id=11298
Yeah, I still encounter IE10 and older sometimes -- looking at the wide audiences Micrio projects are used for, this ranges from grandparent's Windows XP machines to OS-locked government IE10-PCs.
Can't be helped, so an automatic fallback to the previous Micrio version is the least I can do.
I do really hope that there could be a _final_ update from MS to IE10 to at least support ES6. That would also make such a big difference.
Really interesting question. Whereas I don't have power-specific benchmarks, the general rule of thumb for power consumption is indeed the amount of CPU + GPU usage. It's an eternal battle to stay feature rich and keep smooth texture downloads, but use as little power as possible. And ofcourse don't draw any unnecessary frames.
What I managed to do with the entire Wasm rewrite operation, is definitely get the number of CPU cycles down by a lot, and shortening the pathways between raw CPU and GPU operations.
Since this article I was already able to do a lot more optimizations, because of the resulting new code architecture being so much clearer than before. Funny how more minimal structures allow for better optimizing.
Someone else in this post asked the same question and I answered briefly there. Indeed the engine rewrite itself from general purpose to specific usage probably made the biggest difference performance-wise.
However, I do use matrix4 calculations for the 360 viewer, for which Wasm definitely is better.
Interesting idea to do a followup benchmark using the AssemblyScript as TS! I might actually have to do that :D
Thanks for the comment! I agree that the change to a custom tailored engine probably made the biggest difference performance wise. At the end of the article I also briefly mention this.
However, having a monolithic compiled Wasm module which contains all of (and only) the rendering logic is really nice on a codebase-level.
Also, the Wasm part of Micrio is being used for a lot of realtime matrix4 calculations. While I didn't use this specific library, here is a bench page which shows that Wasm really kills JS performance here: https://maierfelix.github.io/glmw/mat4/ .
So it definitely makes a difference. If I had the time I would port the Wasm engine back to JS and do more benchmarking. But that's a big if :)
I can see how WASM definitely improves performance in those benchmarks (where we're talking about thousands of matrix operations). But I imagine your per-frame camera & matrix math is probably not taking up much time (within a 16ms budget), so a lot of these WASM optimizations may not have any real influence on the rendering performance. But, they could introduce more upfront parsing/loading time for your library (i.e. base64 decoding, WASM instantiation), and obviously a bring with them a lot more complexity (in authoring the module and supporting legacy devices).
Anyways, it's pretty cool that you can call GL functions directly from WASM, I hadn't realized that before and it probably would make for an epic game engine to have almost everything called from WASM land.
Regarding the "emscripten-imposed file size problem", I've written a couple of alternative cross-platform headers which might help in this case (e.g. avoiding SDL). These enable WASM apps in the "dozens of KBytes" range, and you get native-platform support "for free":
The binaryen toolkit comes with a wasm2js tool, you could translate the wasm back to js and see how performance compares ;)
It's possible that performance isn't all that different, because asm.js-style Javascript can be surprisingly fast (compared to "idiomatic" human-written Javascript).
Otherwise it's a completely pointless excercise of course, unless you need to support browsers without WASM support (which don't exist anymore AFAIK).
It's unnecessary with AssemblyScript due to AS is just subset of TypeScript and can transpile to ordinal idiomtic javascript. Also AS already uses binaryen under the hood and can produce asm.js via --jsFile, -j command line.
It's really a shame that it simply gives you the original files, not allowing very high res images (ie greater than 10.000 x 10.000). We built a tool for this that's being used by for instance Rijksmuseum and Van Gogh Museum http://www.vangoghmuseum.nl/ , which allows for ultra-res images. The Met should also consider using http://micr.io/ or any similar technology.
Ofcourse the huge volume is great, it's awesome! But by only offering single high-res .jpg files instead of image tilesets just makes the viewing experience less than pleasant.. Downloads are slow, the pages are unresponsive, and I can imagine it's going to cost them tons of unneeded bandwidth.
I agree that "deep zooming" is desirable. Looks like Micrio doesn't support the International Image Interoperability Framework [0], something I think a museum would be interested in. Oh, and I hate that http://micr.io uses a fullscreen background image that highjacks the scroll wheel.
Thanks for your reply, you're right about that. Currently we've only been developing micr.io based on wishes we received, that includes features such as hotspots, tours, etc. See https://tuinderlusten-jheronimusbosch.ntr.nl/en for its first release. We simply haven't had the time yet!
