Threads for jonpalmisc

  1. 7

    How does this compare to binwalk?

    1. 14

      We started with binwalk, but the main reason we developed our own solution is that it was not good enough.

      The biggest difference is that binwalk just goes through a file linearly and whatever it finds, it tries to extract, resulting in a lot of noise (like license text) and false-positives. unblob is way smarter and more precise, by extracting files based on their format specification (like recognizing the format header struct and carves out files based on size values in headers). Here is an example for NTFS: https://github.com/onekey-sec/unblob/blob/main/unblob/handlers/filesystem/ntfs.py#L68

      We are using it for months in production, and the results are way better than with binwalk before. We are getting less false-positives and even if unblob fails to extract everything, we still get meaningful information out of firmwares, where binwalk just failed with no output previously. It’s in feature-parity with binwalk and because of Hyperscan it’s faster and we can handle bigger (4Gb+ firmwares) with no problems, which was not possible with binwalk.

      1. 4

        Thanks, that’s really useful information. The NTFS extractor example is very motivating as it looks very neat. Perhaps this comparison with binwalk would be a useful addition for the README?

        1. 3

          Forgive me for being lazy and not simply reading the source code, but how easy is it to “teach” unblob about additional formats, etc? When using unblob’s API, is it possible to register an additional “format recognizer” that would integrate with unblob or would I have to fork it as a whole?

          1. 12

            It’s very easy, you have to implement literally 1 Python class with 1 method! We have a step-by-step working example you can follow even if you have not much programming experience: https://unblob.org/development/#writing-handlers
            For some formats, you can just copy-paste the C struct from the format specification, calculate the end of the file and run an extractor in one line and that’s it.

            Depending on the format’s complexity, it’s possible to implement support for a new format in a couple of hours!

            We also have a plugin system in place (not documented yet) which we are using in production, so you can just install a Python package and have extra handlers! Pretty neat.

            1. 2

              Thanks for the detailed response. Sounds like you guys have a well-designed system in place. Looking forward to checking it out later today.

      1. 22

        Well, they brought MagSafe back, and the M1 chips are almost unbelievably good, so current state = good. The worst thing I can say is most all their screens are 60 Hz.

        I would recommend the cheapest M2 laptop they offer, and maybe some SD cards (some folks think the storage is small).

        1. 6

          I think all of the M1/M2 macbooks have ProMotion (120hz) displays

          1. 14

            The 14” and 16” M1 MacBook Pros have ProMotion displays. The 13” M2 MacBook Pro and the M1/M2 MacBook Air do not.

            1. 2

              My M1 macbook pro does not have promotion. it’s 60hz, non HDR. still a very good panel though.

            2. 4

              Totally agreed on everything but the “cheapest” if only because the SSDs were changed from 2 separate chips down to 1 which makes read and write performance noticeably slower.

              1. 2

                If you go with SD cards be sure to check the speeds as well, since there are still very slow SD cards being sold.

              1. 8

                Just because no one here has mentioned Safari (that I’ve spotted):

                I use Firefox on MacOS because I want it to continue to exist and, selfishly, it does work really well for me. I have no complaints at all. I don’t notice speed differences when I try other browsers, and I like the small selection of add-ons I use, most of which are probably available on other browsers.

                I don’t like Google’s tracking or their near monopoly on browser engines (ironic as I did some work on the foundations of konqueror once, though not khtml itself) so I avoid Chrom(ium) unless I can’t get something to work in Firefox, which has happened once in the past five years or so.

                Anyone use Safari and swear by it? I have an ad blocker for it which seems to work, and also the 1Password extension, so I could use it, but thanks to M1 and the ability to fully charge my Air from a portable external battery when needed, I’m not concerned about saving battery as much as I was. Is there a reason to use Safari once you know that Firefox exists and don’t mind installing it on each new machine?

                1. 3

                  Safari has always had the smoothest performance for me. It’s the only browser I use. Pedantic complaint, but simply resizing a window has visible lag on Firefox and Chrome whereas I can resize a window at 120 fps under Safari with no visible lag in page layout, etc. I use AdGuard for blocking ads and have had no issues.

                  Been meaning to check out Orion as well, but haven’t been compelled enough to switch just yet.

                  1. 2

                    I’m fairly satisfied with Safari, but I really wish I could have straight up uBlock Origin.

                    1. 1

                      I try to follow the “When in Rome” approach for most native apps, browsers and tools. On my work laptop (Mac), use Safari. At home, use Firefox. On a Windows machine, use Edge or whatever. Same approach goes with (most) tooling configurations, use the defaults as much as possible. As someone who constantly reconfigures Vim, a lot (really… a lot) of time can get sinked into the customization my digital experience. Some things, like security, are uncompromising, but if my goal is to generally get things done efficiently, then reducing my setup overhead, app/tooling ecosystem, and number of cloud services is step number one.

                      I always think back to an old coworker of mine who’s laptop shit-the-bed one morning and by that afternoon, he was back to working, on all channels, on a brand new machine. Of course, cloud backups are a thing, but sometimes it’s easier to be like water

                    1. 7

                      Do not do this. Godot Engine and the applications it creates have no accessibility bus support and the developers don’t see it as a priority. It is immoral to build applications that inherently exclude blind and motion-impaired users, and illegal to sell such software in some jurisdictions. Use native frameworks or a framework with accessibility tooling built in like GTK or Qt.

                      https://github.com/godotengine/godot/issues/14011

                      1. 8

                        It is immoral to build applications that inherently exclude blind and motion-impaired users

                        I agree that accessibility is important, but what about applications that are inherently content-driven, e.g. a photo editor, video editor, animation software, etc.? Godot seems to have a great foundation for building UI applications, and may be a valid tool for the job in many of these cases. I wouldn’t say that making an inaccessible photo editing or animation app is “immoral”; no amount of screen reader support can describe image effects, etc. in a way that would make the app legitimately usable for someone with impaired vision.

                        1. 5

                          This is an obvious exception, yes. Your application is not excluding blind people, since even an accessible application would not make that activity accessible. That said, you still need to make it accessible to people with other disabilities!

                        2. 3

                          It is immoral to build applications that inherently exclude blind and motion-impaired users

                          In reality that’s true for a huge portion of applications build, also ones built with ones that technically use the web, QT, etc.

                          https://github.com/godotengine/godot/issues/14011

                          There’s a follow up for this issue:

                          https://github.com/godotengine/godot-proposals/issues/983

                          While I in general agree that accessibility is lacking, it’s also a reality that affects huge amounts of software. Given that we are talking about a game engine that is not backed by a big corporation (like Qt) and given that Godot is certainly not a major player for UI and there is that open issue and I am sure contribution would be welcome I’d at least see better targets for the statement “It is immoral to build applications that inherently exclude blind and motion-impaired users”.

                          I don’t agree with the author’s take on accessibility either, but I don’t really see why the blame should be put on the a game engine that is completely open source and so far uses the UI framework mainly for its editor.

                          With that said the whole thing also reads more as a “hey, you can do UI with this thing you might not have heard of and it does a lot of stuff right”. At least I don’t think the expectation is that everyone switches away from Qt, SwiftUI and the web which btw. is a target platform.

                          1. 5

                            I don’t agree with the author’s take on accessibility either, but I don’t really see why the blame should be put on the a game engine that is completely open source and so far uses the UI framework mainly for its editor.

                            I agree! I don’t blame Godot at all. I think building desktop apps with a game engine at all is generally a bad idea.

                            In reality that’s true for a huge portion of applications build, also ones built with ones that technically use the web, QT, etc.

                            That don’t make it right.

                        1. 8

                          As usual with accessibility, going really hard in one direction is often not great site everyone. Here’s another article about why you really want to keep grey / lower contrast for accessibility reasons: https://blog.tiia.rocks/web-apps-why-offering-a-low-contrast-mode-makes-you-more-accessible-not-less

                          1. 18

                            If only there were some kind of style sheet, that could cascade in priority depending on where it was defined.

                            There’s no reason for this to be handled by the developers of every single website. This should be handled by the browser. If a user wants high contrast mode, there’s absolutely no reason there can’t be a setting on the client that forces text to a high contrast setting. It’s just numbers in a configuration file. Those numbers can be changed, automatically.

                            1. 12

                              I wish there was some way of saying to browsers, “use modern defaults; I don’t care what they are, or if they change over time; I won’t touch the style; just make the page look good based on the semantic markup.”

                              1. 1

                                could this be accomplished with a stylus style sheet or something similar? maybe an addon that just removes all style tags and links to style tags?

                                1. 4

                                  There are generic Firefox addons like this but they are generally quite CPU hungry.

                                  I end up just doing:

                                  • disabling all custom or web fonts
                                  • setting minimum font size to 12
                                  • setting default zoom level to 120 %

                                  For web apps that I need to use for work (Outlook web app, Jira, &c.) that have bad contrast, I do try to add custom style sheets to fix some text that’s still unreadable to me after all this.

                                  For articles, I can generally use “reader mode” that does switch to black-on-white, since, you know, that’s the best for reading, but that’s generally not helpful on web apps.

                                  In short, I’d be very happy if it was practical to make a stylesheet or plugin to do this, but currently I would say it’s not, or someone would have made one already.

                                  It seems unlikely ranting at web developers will help with this, so I think it would need to be fixed in browsers, However, I see that as an unlikely development, given that e.g. disabling custom fonts is becoming harder and harder, with for example Firefox for Android removing the ability to disable custom fonts.

                                  1. 3

                                    https://github.com/jayesh-bhoot/enforce-browser-fonts is an add-on that disables custom fonts, and it works quite nicely on Firefox for Android if you’re using Nightly or F-Droid that supports custom add-ons.

                              2. 2

                                This is the reality. Firefox has allowed, and continues to allow, forced colors. Go to about:preferences -> Colors, and activate “Manage Colors”. In the menu that pops up, set your preferred colors and set the “Override the colors specified by the page with your selections above” pref to “Always”. This feature is a lifesaver for me, as I deal with overstimulation and can’t stand having a new palette thrown at me every time I open a new page. It’s also replaced my dark mode addon; anything that gets rid of privileged addons is a win in my book.

                                On Windows, you can enable this system-wide with High Contrast mode. Contrary to the name, WHCM isn’t necessarily for high-contrast themes; you can set any palette you want. Every decent program will then receive a forced palette.

                                1. 1

                                  I would like to use this, but it’s regrettably not practical. As an example it prevents me from seeing the upvote arrow on your comment, and whether I’ve voted already.

                                  1. 2

                                    This is something that browsers (all 3? 4? of them) can improve, though, rather than asking a billion web content creators to behave nicely towards any of hundreds of access concerns.

                                    1. 2

                                      Yeaaah…I used to be a fan of the lobste.rs interface before I started learning about accessibility. This is far from the only a11y issue on this site.

                                      I try to avoid complaining things before filing issues properly and leaving constructive feedback, so here are two I just filed:

                                2. 13

                                  Why not just lower the screen brightness?

                                  1. 4

                                    You could, but then you get extra-lower-brightness for other apps/pages which don’t buy into the white/black idea. I don’t think we’ll get the perfect solution either way.

                                    And even if you adjust brightness, it can be a bad experience. I don’t get any actual health/sight issues from brightness or high contrast, but even with everything turned down to minimum on my phone, that medium post is tiring to read because of the white background.

                                    1. 5

                                      So it actually is low contrast that causes the problem.

                                      1. 6

                                        Only if you can adjust all screens to both go down in brightness low enough and without destroying the colour accuracy.

                                        We’ve got a system with at least 3 interdependent elements in it (defaults, preferences, design ideas, hardware capabilities, accessibility limits, …) - you can’t just point so one of them and say that causes all the problems. (Well, you can, but that’s oversimplification and doesn’t solve any issue)

                                  2. 10

                                    …and I immediately had to flip that article into reader mode in order to read it. Which is not to say it’s wrong, but the contrast between the author’s experience and mine is illustrative, and one-size-fits all probably just isn’t going to work here. As she points out there are media queries, but:

                                    @fly suggested in another comment here that this really should just be the browser’s responsibility, not the page’s, and I agree. For articles I tend to flip into reader mode at the first sign of trouble, but for apps I don’t really have that option. But for desktop apps developers mostly don’t have different styles of buttons; they just use the OS’s widget toolkit and accept what the OS vendor has decided. (of course, that’s changing as everything seems to be electron these days anyway…)

                                    1. 10

                                      This is one reason why the APCA (next generation contrast algorithm) recommends against excessively high contrast, especially for dark themes.

                                      It’s not just halation and migrations: overstimulation is another issue that I personally experience quite a bit. Foreground colors that have excellent contrast against dark backgrounds, like yellow, can cause overstimulate if they’re not appropriately de-saturated.

                                      Special palettes that respond to media queries requesting dark/light schemes and more/less contrast are good, but I believe that defaults should also be as accommodating as we can make them; not everyone is okay with the fingerprinting potential of all these media queries. An APCA contrast of ~90 LcP seems to do the trick. You can go lower if you bump up the font size to compensate.

                                      1. 2

                                        Typo: s/migrations/migraines/

                                        s/cause overstimulate/overstimulate/

                                      2. 9

                                        The goal of accessibility design is not making things “great [for] everyone”. It is ALLOWING people to make things great FOR THEM. Some users will have needs that require high contrast. Some users will have needs that require high contrast. Others won’t particularly care at all, and just want things to be pretty. Others don’t care about the contrast because they’re using a screen reader.

                                        You can’t make things accessible by choosing colors. We have to enable users to configure their interfaces with the colors that they, personally, need or want. This has to be done largely at the browser level, although of course stuff like clear, semantic HTML that doesn’t use clever tricks to do things purely visually is a big part of the ask. But developers shouldn’t be forced to worry about colors. We should be forced to worry about allowing browsers to configure those colors.

                                        1. 1

                                          Agreed. I think a better message here would be “use grey text responsibly”.

                                          I often set body text to 65–75% opacity and reserve 100% opacity for headings, etc. It helps build visual hierarchy while retaining a good amount of contrast. This produces both a nice appearance as well as readable content.

                                        1. 2

                                          Likely going to write a tool for backing up Git(Hub) repos en-masse. I want to support wildcards such as apple-oss-distributions/* as well as just individual repos, e.g. jonpalmisc/dotfiles. If anyone knows of something like this that already exists, I’d be interested. I currently have a prototype which is just a bash script, but would like to make (or migrate to) something a bit more powerful.

                                          1. 3

                                            A while back I made a tool tool that does something similar to this, but more focused on a specific user’s repos.

                                            The GitHub API is pretty simple to use, it shouldn’t be too hard to write the program you’re describing.