Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Static files in Go (bouk.co)
90 points by bouk on Aug 10, 2016 | hide | past | favorite | 21 comments


It's quite a common use-case to embed binary data into binaries, so I wonder why more languages don't directly support it with some sort of directive in source code.

Interestingly the X Window System took the opposite approach for its images. Rather than making C accommodate their images, they made their images accommodate C! https://en.wikipedia.org/wiki/X_BitMap and https://en.wikipedia.org/wiki/X_PixMap are both image formats that consist of C code.


Individual platforms do seem to do it. If I understand Windows resources (and I don't really), they are just file blobs munged into the executable and then accessed through a simple sort of namespace.

Seen that way, resources are over-engineered. The basic compiler feature we weant is something like:

    const char blob[] = __include_binary__("blob.dat");

But if I could do that, then probably the next thing I would do is invent some macro-magic to provide a resource-like namespace on top of the feature.


Windows resources were originally built to fulfil two requirements:

- display an icon for an executable

- automatic localisation of application menus etc.

So FindResource() takes three parameters: a type (icon, string, dialog, bitmap, menu etc), a name, and a language ID. That means that changing the language ID can transparently select a different UI layout and text as required.

Meanwhile Explorer (and its ancestor Program Manager) will display the first icon in the resource file as the default program icon.

Resources are just a separate section within the PE executable format.


Thanks for the explanation.

So it sounds like the critical thing that Windows resources have over the thing I outlined is that they and their namespaces are relatively transparent to external tools such as the Program Manager.

The critical thing is that MS could impose a standard. Anyone can write tools to put such a section to their ELF binaries. But it would be useless because then no one would know to read it.


Windows, Mac OS, BeOS, Symbian and quite a few others offer direct support for it, it is just a matter of using the OS APIs.

This was already like that on the 16 bit days.

For example, with Turbo Pascal for Windows, you only needed to do, after generating the resources via the resource compiler

   {$R resource name}
and then make use of FindResource/LoadResource/LoadFromResourceName


You can always (ab)use the linker! [0]

[0]: (shameless plug) http://stackoverflow.com/questions/8923097/compile-a-binary-...


The article doesn't explain why the author didn't contribute to one of the existing solutions to add the only feature they were missing.


These tools are simple enough that its easier to start from scratch than to try an understand someone else's code.


It does explain it though, as they all implement `http.FileSystem` which constrains what tricks you can pull of


Awesome! The one thing that turns me off a little is `ModTime`. I generally avoid incorporating modification times into my build (generally I force them to 1970), but a hash for the ETag would be very welcome.


Yep etags is on my list of things to implement


I've added etag support


A friend of mine implemented something similar[1], but as a pre-processor for the source code. I'll be honest, it's very scary code but I thought it was interesting. :P

[1]: https://github.com/sysr-q/assets


The better tool I know for this use case is fileb0x[1]. It work for all my needs, has some features that the alternatives lack.

[1]: https://github.com/UnnoTed/fileb0x


I'm not sure I understand how using ego ensures that it is fast. Could someone ELI5 please?


<DISCLAIMER> I have never written a templating engine. Anything posted here by someone who has probably takes precedence over what I say here. </DISCLAIMER>

A thing can only be fast relative to another thing. If ego were the only templating solution available to software developers it would be meaningless to describe it as "fast". I have never used ego and do not know how much faster it is than the alternatives (if at all).

However, ego templates are compiled to go source which ends up as machine code in your program that just writes out the strings. I believe most alternative templating libraries parse your template into a data structure at run time and then need to walk that data structure every time you want to render the template in a new context.

So, consider a template like "<div> <h1> {{title}} </h1> <p> {{description}} </p> </div>"

Using the ego strategy the code that has to execute to render it amounts more or less to a linear series of 5 calls to a method that appends strings into a buffer.

Using an interpreted-template strategy, i don't know exactly what that machine code is going to look like but it necessarily involves a lot of consulting the parsed template to see what the next step is, and maybe even a lot of lookups of strings in hash tables.

Aside: In the Java world, the fastest template rendering engines probably compile a template into JVM bytecode which can then be loaded and executed in the current process, yielding what could reasonably be considered the best of both worlds.


I guess it's because the templates don't have to ever be parsed since they're transpiled to Go code at compile-time. It's been a while since I've used Go's template packages but I thought that they're only parsed once no matter how many times you execute it (assuming reuse of the Template struct). So to me, it seems there should really be no difference from just putting the initialization of the template in an init() func.


Nothing is stopping you from doing something stupid like this though:

    func AtRuntime(writer io.Writer){
        templ := template.Must(template.New("blah").Parse(templateText))
        templ.Execute(writer, someStructhere)
    }
Though most sane devs would put the template parsing code in init()


This brings up another question...

Couldn't you just put them in the global namespace in the package? I also don't see the need for init since this is just parsing / compiling a template one time. Maybe I'm just not a sane dev? I just put all my template and regex "must" statements as global vars in my package close to where they're used.


Most definitely, if you want the compiled template loaded for the life of the application, at least. The documentation for template.Must says it is intended to be used exactly like that. If you were in a regular code block, you would check the error and handle it more gracefully.


Ego is as fast as it gets for tenplating, as it compiles the template to Go code. It probably doesn't matter that much performance wise in this case, I would've used it either way.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: