Threads for xcombelle

  1. -1

    sadly enough the author messed javascript object notation (it lack of quote in key of dictionnary) with otherwise syntacly correct python code

    1. 3

      sadly when passing argument arrays become pointer

       void f(char my_array[5]) {
       }
      

      is actually:

       void f(char *my_array) {
       }
      

      at least for my memories of circa 2002

      1. 1
        • Water plants.
        • Polish my little editor/IDE named just o.
        • Try to reach the rank of 2k in the game of Go, at the KGS server.
        • Plan the summer vacation.
        • Update some Arch Linux packages once archlinux-keyring sees a new release.
        • Take the kids to the beach to swim in the ocean, and then play some Mario Kart.
        • Finally start reading the “Artificial Intelligence in Finance” and “Linux device drivers” books I have on my desk.
        • Make a track on the Octatrack, using only self made samples.
        • Do one hangup.
        • Work.
          1. 1

            OGS is also nice! Cool that you reached 2 kyu. Do you remember what you did to go from 3 kyu to 2 kyu? Just play more games, get reviews, read a book, practice life and death or something else entirely?

        1. 1

          unless I mistake the first example show bytes where half-byte change of endianess despite the fact that byte has no endianess usually

          1. 9

            To avoid SQL injection or storing raw HTML in the database, it’s common practise to escape all SQL statements and HTML from the request body.

            looks like someone read a PHP tutorial, even PHP removed magic_quotes on PHP 5.4.0

            1. 1

              I don’t understand how it is supposed to run C program safely

              1. 1

                You hire decent C programmers.

                1. 3

                  So, it doesn’t, then?

                  1. 1

                    As far as I can tell it doesn’t provide any additional safety to the applications. README speaks only about benefits of using Rust only in the context of the kernel development.

                1. 4

                  My website is about 30 lines of shell; I guess I need to brag now: http://len.falken.directory/misc/writing-for-the-internet-across-a-human-lifetime.txt

                  1. 2

                    looks like http://len.falken.ink is a bad link in yout text

                    1. 1

                      Yeah, I refused to pay $40 to renew the .ink domain so I let it expire and bought a super cheap .directory domain.

                      1. 1

                        Considering your old domain is now scammy as hell you probably did the right thing… but you should update old links.

                        1. 1

                          Good point; will do :)

                    2. 1

                      Hard wrapped text looks weird on a screen that’s narrower than the chosen width.

                      1. 1

                        Your site is a little gem!

                        Not only plain text, but some posts are valid Prolog programs!

                        I must admit find it is a little too radical to not have any clickable links, so I find difficult to reproduce your style in my own page; on the other hand, your site is easily read in all the platforms and a browsers I use, including lynx(1)!

                        Very well done!

                        1. 1

                          Double click -> Right click -> open in new tab. Browsers could be smarter and highlight links in text files.

                          1. 2

                            Certainly it is not a problem on the desktop.

                            But that workflow is more complicated on lynx(1), where I have to trigger tmux(1), navigate to the part of the page it has the link, copy it, then paste it on the command line to open a page inside lynx(1).

                            Given you are using plain text files, I imagined you had TUI browsers in mind. It does not seem the case. (In fact, I just noticed your landing page is a RSS feed, which is nicely rendered in the desktop, but not much on mobile, lynx(1) or w3m(1).)

                            Anyhow… I liked it quite a lot! 🙂

                      1. 1

                        if I remember correctly actually it is hack of python when using s += “some characters” if s has only one reference counting, cpython know it can reuse the same buffer

                        1. 1

                          I’m trying to figure out how this differs from what had already been published by the same authors on using AlphaZero to explore the same variant rules.

                          Especially since I’m seeing chunks of word-for-word repeated text from the older paper. Is this new link just a much shorter summary or something?

                          1. 1

                            your link is more complete,

                          1. 4

                            The program above creates a perfect binary tree with 2^n elements and adds them up. Since it is recursive, HVM will parallelize it automatically.

                            The program above runs in about 6.4 seconds in a modern 8-core processor, while the identical Haskell code takes about 19.2 seconds in the same machine with GHC. This is HVM: write a functional program, get a parallel C runtime. And that’s just the tip of iceberg!

                            This doesn’t sound super fast, although if you’re a haskell user this is fantastic, assuming the difference isn’t all start up time. Am I missing something?

                            And yes I’m on record as believing that security concerns of predictive execution will likely force us to models more like this.

                            1. 8

                              GHC programs don’t have start up time, and are also very fast (if well written of course, blah blah).

                              1. 4
                                1. 1

                                  I don’t understand, how can you tell that an exponential program running in 6.4 or 19.2 second “does not sound super fast”? Can you make guesses at how long summing 2^30 leaves should take, or are you comparing with the same program written in another system (which one)?

                                  1. 1

                                    Well let’s say that this benchmark doesn’t impress me except relatively.

                                1. 4

                                  However, when code is optimized, all assert statements are ignored.

                                  Oh my! Thanks for sharing, this is good to keep in mind.

                                  1. 1

                                    actually removing the assert is the only thing that optimisation does

                                    1. 4

                                      No, it also set the __debug__ variable to False.

                                      https://docs.python.org/3/using/cmdline.html#cmdoption-O

                                  1. 5

                                    This is great! I would suggest one way to go even simpler: Use some raw file format for the images, and use a generic file compression afterwards. There isn’t really any reason why compression and image file format need to be coupled.

                                    1. 15

                                      Many image formats are close to some variation of that theme (header + prefilter + generic compressor). From a quick look at qoi it seems like a twist on the LZW family. The prefilter stage here is updating the dictionary based on complete pixels rather than ‘per byte’ (index is xor:ing the channels together). I’d speculate that LZ4 would perform similarly compression ratio wise on planar image layouts. Both suffer when images start to get wide. Two easy improvements now while staying O(n) would be a space filling curve (hilbert) over a quadtree or simple tilemap.

                                      There are plenty of reasons why compression and image formats should be coupled, even when staying lossless. Several generic compressors get greatly improved performance by agreeing on pre-trained sets that are representative of the data to send. These dictionaries need to be agreed upon in beforehand, and thus becomes part of the image “format”.

                                      1. 7

                                        Generic compression can only go so far, to achieve good compression ratios you need to have some real insight into what structure the data has, and how to take advantage of it. I found this post about compressing SPIR-V shaders to be a good example of how those structures can look like.

                                        1. 5

                                          You may be interested in farbfeld from the Suckless folks, which is pretty much exactly what you are suggesting.

                                          1. 5

                                            It depends on the data. Generic compression would certainly do a good job with run-length encoding, as well as with efficient representation of recently-seen values. But method #3, storing colors as deltas, seems like the sort of thing that most generic compression algorithms won’t do. If you were trying to losslessly compress a photo, I would expect QOI to do a better job.

                                            On the other hand, generic compression will also recognize patterns that QOI can’t! If you have lots of dithering, generic compression is probably better.

                                            Either way, it’s remarkable how far simple algorithms can take you.

                                            1. 2

                                              But method #3, storing colors as deltas, seems like the sort of thing that most generic compression algorithms won’t do.

                                              This depends a bit on the image format. For ‘raw’ images, you generally have two possible representations: you store each pixel as a sequence of values one per colour plane (array of structures) or you store each colour plane separately (structure of arrays). Most GPUs support a variety of minor variations on these two themes.

                                              A lot of general-purpose compression algorithms handle small deltas quite well because sequences or values where the entropy is in the low bits are very common. This works well in the structure-of-arrays representation, less well in the array-of-structures representation.

                                              1. 2

                                                png does storing colors as delta

                                              2. 2

                                                There is one big reason: applications want to open and edit files. If all you care about is archiving, then it’s fine to separate these, but go generally want to constrain the set of things that every application that opens images needs to handle. This is one of the problems with TIFF: it’s a family of different compression and encoding schemes and a lot of things implement different subsets of them.

                                                This approach works well with tar because tar is a tape archiver: the file format is designed for streaming access, not random access, and so you can always use an external decompresser and pipe the output to tar for decompression. It doesn’t work when I want to open a file in my photo editor and it doesn’t know how to handle the particular decompressor. This is why most successful file formats specify a small set of CODECs.

                                              1. 6

                                                As far as I know “0e4” == “0e5” us still true

                                                1. 5

                                                  Confirmed:

                                                  $ php -r 'echo (4 == 5) ? "yes" : "no";'
                                                  no
                                                  $ php -r 'echo (0e4 == 0e5) ? "yes" : "no";'
                                                  yes
                                                  $ php -r 'echo ("0e4" == "0e5") ? "yes" : "no";'
                                                  yes
                                                  $ php -v
                                                  PHP 7.4.16 (cli) (built: Mar  2 2021 10:35:27) ( ZTS )
                                                  Copyright (c) The PHP Group
                                                  Zend Engine v3.4.0, Copyright (c) Zend Technologies
                                                      with Zend OPcache v7.4.16, Copyright (c), by Zend Technologies
                                                  
                                                  1. 2

                                                    This is a case where you need the dreaded === (which should be defacto default for most devs these days anyways)

                                                    1. 6

                                                      I tries to explain the problem to lavarel dev and did not succed to explain the problem see https://github.com/laravel/framework/issues/20141

                                                      1. 2

                                                        Well it’s certainly not PHP’s fault if people are actively averse to learning how to properly use it.

                                                        It’s been a while, but even 10 years ago we would have never accepted someone saying “well 0 and 0 are somehow equal” like one person in the moved bug tracker link does. It’s either a string or an int, otherwise it’s a bug. Just because the language is not strongly typed that doesn’t mean you shouldn’t know with what you’re working inside your application (and not where any input enters the system).

                                                        1. 2

                                                          Oh wow, did not expect people to not use strict comparisons.

                                                      2. 1

                                                        And it will never be changed, which is a good thing if you value BC.

                                                        1. 1

                                                          I mean that == should be deprecated somehow. with a big enough warning

                                                          1. 1

                                                            BC?

                                                            1. 2

                                                              Backwards compatibility.

                                                              1. 1

                                                                Backwards Compatibility. The thing that everyone outside PHP decries so much - e.g. fixing all the needle/haystack and 90% of other “oh if only it were different” things. Because old PHP code is actually really easy to run on modern versions.

                                                                1. 2

                                                                  I honestly don’t have a horse in this race. But even just at a glance I don’t think it’s an “outsider” vs. “insider” matter. It doesn’t have to be so partisan. Both have upsides & downsides.

                                                                  It would be nice, for instance, if there were an option like PHP, but with a balance between BC and solid interface design. Maybe Python or Go represent this, who knows.

                                                                  I’m open to other perspectives, but my point is as follows: a decent interface doesn’t have to be made great, but a trashfire interface should definitely be made at least decent before it’s “frozen”.

                                                                  1. 2

                                                                    I think that’s a valid standpoint, but there’s a reason people today usually make a big fuzz about a 1.0 release from which point on it’s declared stable/non-breaking.

                                                                    But if we ignore PHP 1+2 and only look at 4.0 which was released in May 2000 - that’s when the “we don’t break BC” decision was made and it’s been held up since. So that’s why I think “they should’ve” is not so helpful. They didn’t and there are always good reasons for and against.

                                                                    In case you’re not so familiar with the history, there historically was a huge problem of getting the web hosts to upgrade DESPITE the massive efforts to keep BC. IIRC on August 8th 2008 there was a huge effort to promote this deadline (4.4.9 was the last PHP 4 version that was released) to get the people to finally move to PHP5, which was released 4 years earlier in summer 2004 and AGAIN so many people didn’t upgrade (webhosts and wordpress was a big factor here).

                                                                    I personally would also agree that there was a point (maybe 7.0) where it should’ve been done, but on the other hand I am personally guilty of being one of the people going “our dependency libraries are not available for Python 3, let’s start this massive new project in Python 2 in the year 2014”.

                                                          1. 3

                                                            Looks like custom query is a new attack surface

                                                            1. 3

                                                              How so? Isn’t this proposal just a way to standardize what we already do in some other ways?

                                                              1. 1

                                                                There are two major issues I can see:

                                                                1. developers will need to consider this continuously, and they already have issues ensuring that all verbs’ paths transit security controls
                                                                2. compensating/mitigating controls at the edge will need to consider this, and they’re notoriously bad at keeping up to date with standards.

                                                                The first is relatively straight forward: developers will now need to track a new verb that they have to make sure that all authentication, authorization, &c controls are applied. For frameworks that tend to have a single object/method per URL/route (endpoint), this is relatively straight forward: you can have a single entry point for the endpoints, and just detect if the new verb is being used to access it. For frameworks that tend to separate out endpoints, this means we need a new method, it needs to have all controls applied, tested, &c. It’s not a huge deal, but it’s often an edge case I see when working with developers, but also not too different from our current pain.

                                                                The second is more myriad; generally, depending on your devices, servers, security software, &c &c, you can have all sorts of odd interactions between these devices and the destined applications. For example, years ago many large venders had very strict rules for SQLi and XSS. So how would we bypass them? well, we’d just change GET or POST to Foo (or the like). At the time, several application servers would use heuristics for verbs to determine how they should be interpreted: if it had a body, it was a POST, if not, it was a GET. But how did this bypass edge controls? Well they also had heuristics: if they could determine what the verb was, they would apply rules, otherwise they would just pass it to the downstream application server. If you application server was reliant on the edge controls to block SQLi or XSS, you were in trouble. We’ve gotten much better about these sorts of things, but they can still lag significantly (HTTP/2 controls come to mind, or SCTP as a TCP replacement for data exfiltration, because many systems simply pass those along).

                                                            1. 15

                                                              Seeing the examples use floats for currency made my eye twitch uncomfortably. None of the code I’ve written for financial institutions did that. Is that really done in this space?

                                                              1. 37

                                                                I used to work at a firm that did asset valuation for bonds, and we used floats :).

                                                                It’s generally fine to use floats when it comes to asset valuation, since the goal is to provide an estimate for the price of a security. Our job was to estimate the value of a bond, discounting its price based on 1) the chance that the issuer will go bust or 2) the chance that the issuer will pay off their debt early.

                                                                My understanding is that floats are never used in places where a bank is dealing with someone’s physical assets (it would be a disaster to miscalculate the money deposited into someone’s account due to rounding errors). Since our firm was not dealing with money directly, but instead selling the output of statistical models, floats were acceptable.

                                                                1. 9

                                                                  That makes absolute sense to me. Thanks for sharing the difference. We were dealing with transactions (and things like pro-rated fees, etc.) so even for things where it made sense to track some fraction of a cent, it was “millicents” and integer arithmetic. I wasn’t thinking in terms of model output.

                                                                  1. 4

                                                                    it would be a disaster to miscalculate the money deposited into someone’s account due to rounding errors

                                                                    IME the really really hard thing is that summing floats gives different answers depending on the order you do it in. And summation operations appear everywhere.

                                                                  2. 7

                                                                    @jtm gave you a bit more detail, the original post offers this in the Other notes section:

                                                                    One of things that tends to boggle programmer brains is while most software dealing with money uses multiple-precision numbers to make sure the pennies are accurate, financial modelling uses floats instead. This is because clients generally do not ring up about pennies.

                                                                    1. 6

                                                                      Ah I missed this, but yes – exactly this.

                                                                      This is because clients generally do not ring up about pennies.

                                                                      An amusing bit about my old firm: often times, when a bond is about the mature (i.e. the issuer is about to pay off all of their debt on time), the value of a bond is obvious, since there is a near-zero chance of the issuer defaulting. These bonds would still get run through all the models, and accrue error. We would often get calls from clients asking “why is this bond priced at 100.001 when its clearly 100?” So sometimes we did get rung up about pennies :).

                                                                      1. 2

                                                                        If that was there when I read it, I overlooked it because my eye was twitching so hard.

                                                                        1. 2

                                                                          It’s completely possible they added the Other notes section later! Just wanted to share since it addressed your question directly.

                                                                      2. 3

                                                                        I never wrote financial code, but I also never understood the desire to avoid floats / doubles. They should have all the precision you need.

                                                                        Decimal is a display issue, not a calculation issue. I think the problem is when you take your display value (a string) and then feed it back into a calculation – then you have lost something.

                                                                        It’s like the issue with storing time zones in you database vs. UTC, or storing escaped HTML in the database (BAD), etc.

                                                                        Basically if you do all the math with “right”, with full precision, then you should be less than a penny off at the end. I don’t see any situation where that matters.

                                                                        Although on the other side, the issue is that “programmers make mistakes and codebases are inconsistent”, and probably decimal can ameliorate that to some extent.

                                                                        I also get that it’s exact vs. inexact if you advertise a 0.1% interest rate, but I’d say “meh” if it’s a penny. It’s sort of like the issue where computer scientists use bank account balances as an example of atomic transactions, whereas in real life banks are inconsistent all the time!

                                                                        1. 11

                                                                          I also never understood the desire to avoid floats / doubles.

                                                                          Addition isn’t associative, so the answers you get from summations are less predictable than you would like

                                                                          1. 7

                                                                            I think in practice the issue may actually be that floats can be too precise. Financial calculations are done under specific rules for e.g. rounding, and the “correct” result after multiple operations may actually be less mathematically accurate than if you’d just used 64-bit floats, but the auditors aren’t going to care about that.

                                                                            1. 4

                                                                              It’s not just that, it’s that the regulations are usually written to require that they be accurate to a certain number of decimal digits. Both the decimal and binary representations have finite precision and so will be wrong, but they’ll be differently wrong. Whether the binary floating-point representation is ‘too precise’ is less important than the fact that it will not give the answer that the regulators require.

                                                                            2. 4

                                                                              like @lann and @david_chisnall mentioned, it’s not about being precise, it’s about getting the answer expected by the accountants and bookkeepers and finance people. Back when they were doing it all on paper, they built certain rules for handling pennies, and you have to do it the same way if you want to be taken seriously in the finance/banking/accounting industries. Back then they couldn’t cut a physical penny in half, so they built rules to be fair about it. Those rules stuck around and are still here today and are sometimes codified into law[0]

                                                                              As for “meh” it’s a penny, they generally don’t care much about anything smaller than a penny, but they absolutely care about pennies. I regularly see million dollar transactions held up from posting because the balancing was off by 1 penny. They then spend the time it takes to track down the penny difference and fix it.

                                                                              0: PDF paper about euro rounding

                                                                              1. 1

                                                                                how do you store 1/3 with full precision ?

                                                                                1. 1

                                                                                  Not with a decimal type either :)

                                                                                  1. 1

                                                                                    sorry I misread your post

                                                                            1. 1

                                                                              unless I mistake running anything you want without excalation of privilege is still a major concern, for example a ransomware will do a major security concern

                                                                              1. 3

                                                                                The article was written in (2006) - not even browsers did any real privilege separation back then. The world has lost its innocence quite a few times since then.

                                                                              1. 1

                                                                                If I don’t make mistake the first example of the code in documentation ( https://paranoid-scientist.readthedocs.io/en/latest/ ) a failure to correct answer if x<-1 or 0<x<1 because in those case x**3 < x

                                                                                1. 1

                                                                                  the test is saying {if x >= y then x^3 >= y^3}, not {x^3 >= x}

                                                                                  1. 1

                                                                                    I was wrong

                                                                                1. 2

                                                                                  The argument that the code is written for python2.7 is pretty poor seems just print statement is requiring python2 adding a pair of parenthesis is not so hard

                                                                                  1. 15

                                                                                    Its not just the print statement that changed between Python 2 and 3. There’s a number of subtle differences that might affect the semantics of a program, and I wouldn’t want to debug those in someone else’s project I was using to quickly check one thing.

                                                                                  1. 4

                                                                                    looks like an advertisement for fastmail

                                                                                    1. 2

                                                                                      Well, I like the service. :-) I’m not affiliated with the company in any way, but I felt like sharing my positive experience with them.

                                                                                      1. 3

                                                                                        you give an affiliation link at the end of your post. that’s look like a kind of retribution

                                                                                        1. 3

                                                                                          Fair point. It didn’t cross my mind that someone might interpret it like this, because I definitely don’t care about 0.50$/month, but I guess I was mistaken. I’ve removed the link.

                                                                                          1. 3

                                                                                            For every user you refer to Fastmail via your special link, you receive free credit on your account, and they receive a 10% discount for a whole year!

                                                                                            looks like a lot more than $0.50/month when you multiply by user

                                                                                            source:

                                                                                            https://www.fastmail.com/help/account/referrals.html

                                                                                            1. 7

                                                                                              Still, I don’t think anyone ever got rich from Fastmail referrals. :-) (so far I’ve referred a grand total of 0 people there) My point was really to offer some small savings to people interested in Fastmail, nothing more.

                                                                                    1. 4

                                                                                      Pleased to find that Sandstorm blocks this, by two different mechanisms (no /proc and no userns).

                                                                                      user namespaces in particular introduced a massive amount of new attack surface, and when it came out there was a flurry of privilege escalation vulnerabilities related to it – all sorts of kernel interfaces that were previously only accessible by root (such as unmount() in this case) that all of a sudden are potentially exposed to untrusted users.

                                                                                      1. 2

                                                                                        Does Sandstorm block the attack or only the POC ?

                                                                                        1. 2

                                                                                          IIUC it should block all forms of the attack – this requires being able to access virtual files backed by seq_file, and we don’t expose any of those to apps – just disk files and /dev/{null,zero,{u,}random}

                                                                                          Edit: we do bind-mount /proc/cpuinfo, which I’m not sure whether it’s backed by seq_file, but the app shouldn’t be able to affect its contents, so I would be very surprised if that changed anything.

                                                                                        2. 2

                                                                                          You’re correct, but I think it’s important to add that in the long term user namespaces will enhance security a great deal.