1. 2

    mmm nice just went through trying this out the other day.

    One point to mention is that they are great on large scale data where you don’t want to send MBs of GeoJSON to the client.

    Here is the output I made to visualize potential broadband connectivity - http://broadband.cat/mississippi.html

    1. 2

      they are great on large scale data where you don’t want to send MBs of GeoJSON to the client.

      this is a great key point worth emphasizing, you don’t only save on the data transfer, but also in performance, both to parse and to render a giant GeoJSON file.

      Nice work, thanks for sharing!

    1. 1

      I’ve done the ASUS ac-1900 reflash and it works nicely. You can pick these up cheap as t-mobile devices.

      You can also combine multiple or a 86u for mesh wifi and cat5 backhaul work.

      https://www.bayareatechpros.com/ac1900-to-ac68u/

      1. 2

        Interested to know what features made this the choice. Tried pybind and it seemed OK too.

        1. 1

          I’m interested too! I haven’t gotten around to doing a proper comparison yet myself due to other deadline pressures, but I owe my coworkers one soon. Happened upon this article while researching options. This article indicates that cppyy works well with templates, maybe that’s the killer app? I look forward to experimenting with the options but pybind was another top contender for me. My team has a mixed C++/Python codebase so we could definitely use a binding solution that works both ways. We haven’t had a pressing need for it quite yet though.

          1. 2

            Nice thanks for the link.

        1. 6

          Beautiful Visualisations here. I wonder if using non-smart phones can help in anyway.

          1. 3

            According to the article the data comes from a company collecting precise movements using software slipped onto mobile phone apps. So maybe installing/using less apps helps somewhat…

            1. 4

              After reading this, I deleted half the apps from my phone and disabled location services for most of the ones that were left.

              1. 2

                I would assume that disabling location permission for apps would also be sufficient. I can understand why maps needs my location, but not much else.

                That said, the article does say the weather channel was also at this, and I guess a lot of people would give this more trust as its a default app on ios.

                1. 5

                  Yes and no.

                  The wifi MAC address method is also becoming quite prevalent. If your phone has wifi just enabled it can be pinged. [1]

                  Then over the cell network your signal can be triangulated.

                  1. https://techcrunch.com/2019/05/22/mind-the-privacy-gap/
                  1. 2

                    True, thats what skyhook does as well. Interestingly (and this might show a conflict of interest within Google) Android has started using random mac’s for wifi. https://source.android.com/devices/tech/connect/wifi-mac-randomization

                  2. 2

                    the weather channel was also at this, and I guess a lot of people would give this more trust as its a default app on ios.

                    Is it? I know there’s a weather app by default, but I thought the Weather Channel app was a separate download (not a regular iOS user here). I uninstalled The Weather Channel on my Android device after they were outed as scraping contacts and selling that off. No reason to install anything of theirs after that and now this as well.

              1. 1

                The bar has been set. This is clearly a leader board… first developer to make #1 in all categories, can claim the “glory”.

                1. 1

                  Interesting write up and great to learn about where the various projects are going. Has anyone tried computationally intensive tasks with this in the wild? This mentions some: https://stories.dask.org/en/latest/

                  Also wondering about comparisons to https://github.com/modin-project/modin

                  1. 3

                    I used Dask at previous job, for multiple steps of gene sequencing pipeline. Worked well for us.

                    Modin is similar to Dask’s Pandas support, I think. However, Dask is much more general purpose than modin: while it does have Pandas Dataframe emulation (and numpy array emulation), you can also run arbitrary calculations on arbitrary data, which is what we did: https://docs.dask.org/en/latest/bag.html

                    Also, Dask has very nice single-machine mode. Possibly ray (which modin uses) has this too, not sure.

                    1. 2

                      | Also, Dask has very nice single-machine mode.

                      I agree, compared to something like Celery it’s a much nicer development experience (of course, use-cases are slightly different). No need to setup any external services like message queues etc. And you can even choose between creating a local cluster by using processes, or threads. I’ve started to use Prefect recently, which uses Dask under the hood. I was very pleasantly surprised how little friction there was before I had something usable.

                  1. 5

                    Holiday is over, back to work. We have a new machine with two RTX 5000s and plenty of memory/cores. Our project was running against the limits of our Tesla M60 (dual GPU) and Tesla K20c (an oldie, still good for training feed-forward networks or lighter RNNs). So, there is ample capacity again for new experiments. We also have two new Radeon VIIs, but these are still being set up.

                    I have a local Hydra instance set up for automatically testing models using some Nix on git commits, but I still have to deploy that properly on a work server (which requires some hoop jumping due to CentOS 7, storage via NFS, etc.).

                    Also writing up some recent work.

                    1. 1

                      any more info on the lab setup or development process? Sounds like you are upto some interesting model creation?

                      1. 2

                        any more info on the lab setup or development process?

                        It is mostly quite boring. We use GitHub, do PRs + reviews for everything Travis-CI for CI. Some projects are built using Nix in CI because it is easier to pull in all the dependencies (libtensorflow, Python Tensorflow module, various Rust versions, etc.). The team consists of more senior staff, PhD students, and students. I am very happy that we have been able to grow a team that it both strong scientifically, but also has a good Rust and Tensorflow background. Some public projects are e.g. our Rust packages for training and using embeddings:

                        and a neural sequence labeler:

                        https://github.com/danieldk/sticker/

                        Sounds like you are upto some interesting model creation?

                        Some recent work:

                        And a lot more stuff cooking ;).

                        1. 1

                          Excellent!

                    1. 2

                      Some great advice I got was its a combination of three things and you have to be happy with it. Each of these may have non tangible things beyond financial aspects, work flexibility etc.

                      1. What you think you should be paid
                      2. What the company thinks you should be paid
                      3. What the market is paying

                      Now given these survey the market, the best way I have done this is using the Visa application data for the US. It’s very transparent… https://h1bdata.info/index.php

                      1. 6

                        I’ve just started my summer holidays.

                        Is there any suggestions from lobsters on what to do in Basel, Geneva or easily accessible areas around them?

                        1. 3

                          Basel

                          Well I’d say get up in them mountains! Check out the villages and the Jungfrau.

                          1. 2

                            The Bodensee is really nice.

                          1. 1

                            Zweig Media’s JS-based linear programming solver has been available on the web since 2007: https://www.zweigmedia.com/simplex/simplex.php?lang=en

                            1. 1

                              Oh thanks, will check it out. Any idea about how it compares to alternatives, functionality and speed wise etc?

                              1. 1

                                No, but if you learn anything on that front, I’d be interested to hear about it.

                            1. 6

                              Perhaps put together a vertical herb garden. My last project was a small (2’ x 2’) planter bed which went well, (except I could have used more screws perhaps) so I want to try this slightly more challenging project.

                              1. 3

                                NIce! Definitely share the results

                              1. 3

                                I’m off to Dublin, Ireland for the Euro30 optimization conference.

                                Anyone going or got some recommendations?

                                1. 1

                                  Oh no not my secret recipe for…

                                  looks through generated recipes

                                  Super easy homemade water

                                  1. 1

                                    uh oh I shattered the jar with the boiling vanilla salt mess…

                                  1. 1

                                    I have the dreaded live coding exercise in a few days - I have been thinking about recursion and the Fibonacci sequence lately - thanks for sharing.

                                    1. 2

                                      Here is a great run down of that example - https://hplgit.github.io/primer.html/doc/pub/half/book.pdf

                                      1. 1

                                        What’s that?

                                      1. 2

                                        I want to install an alternator in my Toyota, first time doing that job. Also want to clean my HVAC coils. I’ll probably catch up on GSL (StarCraft II), watch Solo, cancel HBO, start Wolfenstein II. Read more Mastering PostgreSQL, that or a Python ebook. As long as I get that alternator done, I’ll be happy. Advice for a diy mechanic welcome.

                                        1. 1

                                          Cool! Step 1 disconnect battery.

                                          Everything else will depend on make and model ;)

                                        1. 1

                                          Getting ready for my conference talk at INFORMS.

                                          1. 3

                                            How about maps? Anyone got pet peevs about it? I know Open Street Map (OSM) is OK. But also for sharing locations and routes plus calendar integration it’s hard to deplatform (maybe not a word?) from “google”.

                                            1. 1

                                              maybe not a word

                                              A word, but usually used for like, banning some unpleasant hateful person from a platform.

                                              1. 4

                                                Here are my 2 cents:

                                                What are the things you look for in an algorithmic open source project?

                                                I look for how well it is documented. If it’s a library, the API must be documented (‘just read the source’ is a silly thing to tell users who might want to use the thing you made, for example). If there are examples of how to use your thing, that’s even better!

                                                I also look for how active the project is, how commits happen over time, how responsive the developer is to user questions/issues, what the release cycle is like, what type of testing (if any) is performed prior to making releases, and what the project license is (and if it’s compatible with the license I chose for my application).

                                                Are there any assumptions that get made that make a project less attractive?

                                                See the first response about documentation :P. Other than that, other things that might make me walk away are (in no particular order:

                                                • if the developer seems generally hostile towards users reporting issues or trying to make contributions

                                                • if the project has lots of reported issues (that seem like issues I would care about) but it appears the developer has lost interested and is either not resolving them or is not addressing community attempts to resolve them by reviewing/merging patches from others

                                                • if the developer does not pick a license for the code (which in may areas I guess it defaults to ‘look but don’t touch’), or if they pick a license I do not agree with, or if it’s a library they pick a license that is incompatible with the license I chose for my application.

                                                • if there’s little or no testing of code before releases are tagged

                                                What are your best practices?

                                                • Document all the things (I’m no expert here, but trying to improve)

                                                • Not all users are as familiar with your thing as you are, so don’t assume they are if their questions seem too ‘simple’ or ‘basic’. Everyone has to start from somewhere, and some users are more experienced in learning on their own than others. Try to be nice :)

                                                • Don’t let open issues linger. If you plan to fix/address it, say so. If it’s not relevant, close it. If you planned to address it but no longer plan to, say so and close it. Re-opening issues is cheap. You can also publicly ask for help with issues, but this may not always work (kind of depends on how much you might be willing to help newbies get familiar enough to contribute, etc, and maybe some luck..)

                                                • Good unit tests also helps tremendously. Contributors can easily see when something they want to contribute breaks stuff, and in some cases it can even serve as examples for how to use your thing.

                                                • Pick a good open source license that reflects your goals for the project and protects rights of users who contribute to your project. I’m partial to copyleft-leaning licenses (e.g. GPL), which attempt to make sure forks of the thing continue to be open source. Others prefer MIT-style licenses. There are lots, it helps to do some research just so you know what is out there.

                                                Would you rather release it now (in an OK state) v. later (in an excellent state)?

                                                There’s no better time than now! Document the shortcomings (e.g. in a TODO, readme, or by opening bugs/enhancements in your tracker of choice). You can also use things like semantic versioning to convey to folks that your thing is not ready (e.g. 0.X version), or when you introduce breaking changes. If your thing is super cool and you get a lot of folks interested, some may even help move it forward. Don’t count on this though, but it is a nice bonus sometimes when it does happen. There’s a 0% chance of getting this bonus if you don’t release anything publicly.

                                                1. 1

                                                  Thanks, I like the idea of utilizing a tracker for the work to be done.

                                                  Also agree with the taking feedback from any user at any level in a positive way to be a good start.

                                                1. 3

                                                  What are the things you look for in an algorithmic open source project?

                                                  Not restricted to an algorithmic project but:

                                                  1. Documentation (including a quickstart)
                                                  2. Examples
                                                  3. A way to contact the author and report issues and ask questions

                                                  Are there any assumptions that get made that make a project less attractive?

                                                  No documentation, no install instructions

                                                  What are your best practices?

                                                  1. Include a test suite + CI
                                                  2. A few lines on what you want to do next
                                                  3. A brief intro for who the library is targeted at
                                                  4. Pick a license you are comfortable with.
                                                  5. You can get a DOI for it, if your audience is academics

                                                  Would you rather release it now (in an OK state) v. later (in an excellent state)?

                                                  Release it now, mark it as Beta. Get feedback and improve. Ideal outcome is a few other people who understand the project and submit fixes and features.

                                                  Best wishes for the journey! Post the link when you are done!

                                                  1. 2

                                                    Thanks for outlining the key points.

                                                    Interesting point around the DOI, I’ll be publishing a couple of papers that I have been able to write using the code so just planned on listing them. Any idea which is better?

                                                    1. 1

                                                      I’ve seen papers refer to github, but we know this can be ephemeral in the longer run. What we need is some kind of government agency for archiving code. In answer to your question, it’s nice and easy to get a DOI. Why not do both - get a DOI and reference it in the papers and link the papers in the documentation.