1. 3

    This issue is more generic than Python… the true problem comes from piping STDIN and STDOUT from a subprocess.

    One other alternative is to make sure that you are always streaming the STDOUT and STDERR of a subprocess to a dedicated file rather than to subprocess.PIPE.

    1. 1

      Yes you are absolutely correct, the place where I ran into this was with python but it can certainly happen wherever you are piping STDIN and STDOUT and not properly handling the data. In python even thought the documentation does warn about this it still seems like an easy bug to introduce and can be weird to debug.

      1. 1

        The issue isn’t too hard to solve by poll()ing the FDs. The deadlock is an implementation issue.

      1. 9

        Scott Manley has a good video on the EM Drive with his thoughts on the implications and the likelihood that these findings will be substantiated.

        1. 4

          I much preferred this as a ‘rebuttal’ - lots of good information/analysis, and why to be skeptical and what it would mean if it is experimentally proven. Without all the name calling.

          1. 1

            Scott Manley is a great blend of science educator and Kerbal Serial Killer/Space Vehicle Designer.

        1. 2

          Wish git had this built in.

          1. 6

            Probably never will because git is and should be agnostic to what you’re actually using to host the repositories (git != github). There are a number of facaded on top of git (hub for github for example) that act as a pass through for git commands and add extra functionality on top. In my opinion thats the best way to do that sort of thing.

            1. 1

              You are correct, I wasn’t thinking of that, they will not accept it.

              1. 1

                It can make a best guess though and try to translate a git:// url into a http(s). Many git servers also serve a website. Git could even have a config variable for the url to use.

                1. 1

                  Sure it could but that can be said about pretty much anything. The unix philosophy is to have software that is minimalist and does its own job extremely well which I believe that Linus and git are looking to follow. Adding a facade on top of git give the most flexibility and keeps the core application as specific as possible so it can just focus on doing that one thing very well.

              2. 1

                Yes! A friend of mine told me to make a pull-request to github and see if they accept it =)

              1. 2

                Awesome that this looks like it supports both github and bitbucket. If you are only using github take a look at hub which provides a facade over git and adds a bunch of github related features like hub browse which will open the current repo in the browser.

                1. 1

                  Yes, it works on both, github and bitbucket and I will test with gitlab. I will take a look at hub, thanks for sharing.

                  Although my idea is to keep git-open as simples as possible, maybe I will take some ideas and implement them =)

                1. 6

                  Please remember the software tag for release announcements! :)

                  Also, it’s great to see that the Elixir folks are introducing reasonable time support. The Erlang time tuple is a little awkward to work with, and having a bunch of folks introducing their own not-quite-compatible types (Ecto.Datetime, Timex, etc.) is not a long-term solution.

                  1. 4

                    Sorry about that! First time posting, I’ll remember for next time.

                  1. 6

                    It’s not clear if the experiment broke ethical or even legal boundaries, since it relied on confusion if not outright deceit to trick people into installing something other than what they intended to install. Still, the lesson the experiment imparts is worth heeding.

                    I’m not sure I understand what the “ethical or even legal boundaries” they are implying were broken here. It doesn’t go into detail about what the script he wrote does, if it was something malicious that would make more sense. But if i read it correctly he basically wrote a script that shows a warning message telling the developer their mistake and pings home to register the download in order to see how large the attack vector was. Am I missing something or is the article trying to make things sound way more interesting than they really were?

                    1. 6

                      I think the article is just being bombastic. I don’t see anything unethical about it, it’s basically how all computer security research works.

                      That being said, judging from the recent CFAA cases, it probably would be considered illegal by US law. It’s a good thing the student lives in Germany and not the US, or they might be looking at jail time (especially since the package infected .mil domains).

                      1. 5

                        I would argue that the specifics of what he did were both unethical and illegal. Illegal by the letter of the Computer Fraud and Abuse Act, as you mentioned (he certainly exceeded authorized access on the machines that downloaded his fraudulent packages, as the users no doubt had no expectation that downloading the packages would result in searching of their machine or transmission of data to some outside location). Unethical because his packages scanned the user’s machines, including command history, resulting in potential accidental disclosure of private information. I understand that he had some personal justification for this in the context of his research, but without permission (which would likely have had to have been given by the users when they first accessed the package manager, likely with some sort of credential system to track their having opted in to experiments that may expose personal information), this definitely seems like a breach of reasonable ethical practices in the security field.

                        1. 2

                          Where does it say his program scanned the user’s machine? All I got from the article is that it logged its own invocations.

                          1. 4

                            It’s on page 23 of the thesis for which this work was done. Here’s the quote listing what the fraudulent packages collected and transmitted back to the university machine. Note that all data was transmitted unencrypted over HTTP as the query string of a GET request.

                            • The typosquatted package name and the (assumed) correct name of the package. This information was hard-coded in the notification program before the package was distributed. Example: coffe-script and coffee-script (correct name).
                            • The package manager name and version that triggered the operation. The package manager name was also hard-coded, before the package was uploaded. The package manager version was retrieved dynamically. Example: pip and the outputs of the command pip –version
                            • The operating system and architecture of the host. Example: Linux-3.14.48
                            • Boolean flag, that indicates whether the code was run with administrative rights. Getting this information on Windows systems is not trivial and possibly error prone.
                            • The past command history of the current user that contains the package manager name as a substring. This information could only be retrieved from unixoid systems, because Windows systems do not store shell command history data. Example: Output of the shell command grep “pip[23]? install” ~/.bash_history
                            • A list of installed packages that were installed with the package manager.
                            • Hardware information of the host. Example: Outputs of lspci for linux. On OS X, the outputs of system_profiler -detailLevel mini were taken.
                            1. 4

                              OK, yeah, that’s definitely crossing a line.

                          2. 1

                            users no doubt had no expectation that downloading the packages would result in searching of their machine or transmission of data to some outside location

                            Why do you say that? That’s what pretty much every ruby gem I’ve ever installed did. Digs around on my hard drive for a while, downloads some more pieces, compiles some code, runs some code, blah blah, finally announces it’s done.

                            1. 3

                              I suppose I should have been more precise. The type of data collection the program did, in particular greping for commands in the bash history of any Linux machine for commands containing the name of the package manager, and then transmitting the result of that search back to a remote machine is probably behavior the average user would not expect.

                              1. 1

                                A Ruby gem that computes and installs dependencies is not even remotely the same thing as what happened here.

                                I absolutely do not expect installing a package or gem will scrape arbitrary information from my system and send it to an unknown third party, and I don’t think many people do expect that or think it’s okay.

                        1. 3

                          This looks really cool and is one feature that I think is missing from Dash. There are tons of integrations for editors with Dash and its a great resource to be able to quickly look up documentation when you need it but its definitely missing a curated set of examples depending on the documentation you’re looking at, i’ve found most documentation that I use at least is pretty void of usage examples. This means generally if i want to find how its used I’ll need to go over to Google to search for actual usages.

                          I like the way Dash does the integration better than having an always on window that updates while I type, personally I would get really distracted seeing the right side of my screen always flickering with new information.

                          1. 3

                            Sourcegrapher here. Thanks for the kind words. We built this editor integration because we 100% agree with you—actual usage examples are super valuable.

                            You can turn off the live-updating, always-on behavior in the editor plugin. See https://github.com/sourcegraph/sourcegraph-vim#vimrc or https://github.com/sourcegraph/sourcegraph-sublime#auto. Then you just use a hotkey to jump to usage examples. Or you can just keep that browser window in the background (that’s how I use it, since I use a full-screen WM on Linux).

                          1. 2

                            I recently found regex101 and have been using it for all my regex needs. They have some really nice debug information in the right hand side complete with explanations.

                            1. 4

                              Looks like an awesome project, best of luck getting funded! For anyone who is using chrome as their main browser, I’ve used the Vimium[1] chrome extension with some luck. I’m curious if you’ve seen/used this extension before and some of the benefits of qutebrowser over this extension (beside native support for the vim bindings which I would think lends itself to a more fluid experience)

                              [1]https://chrome.google.com/webstore/detail/vimium/dbepggeogbaibhgnhhndojpepiihcmeb?hl=en

                              1. 4

                                Vimium (which I used for a longer time before starting qutebrowser) is mostly about keybindings, while mostly keeping the Chrome UI (it has no other choice, with Chromium’s plugin API). It doesn’t have things like a real commandline, easy extensibility, or a minimal UI.

                                I think the user interface is really important - I have a relatively low-resolution screen (1366x768), and I don’t want a big address/tab bar I almost never look at.

                                Also, with qutebrowser you can do things like :spawn mpv {url} to simply launch mpv to play the current URL. Or :hint links spawn mpv {hint-url} to do the same via hints. Or :download-open to simply open the file you just downloaded. Or edit form fields with e.g. vim by using Ctrl-e.

                                From my point of view, qutebrowser compared to Vimium is basically like vim compared to some IDE with really bad vim emulation.

                                1. 2

                                  Thanks so much for the run down, qutebrowser sounds awesome. I was sad to see that homebrew dropped QtWebKit as I was exited to give it a try.

                                  Using qutebrowser with Homebrew on OS X is currently broken, as Homebrew dropped QtWebKit support with Qt 5.6. I’m working on building a standalone .app for OS X instead, but it’ll still take a few days until it’s ready.

                                  1. 3

                                    I built a standalone .dmg/.app for qutebrowser just a few hours ago, I’ll release a v0.6.0 dmg once some people confirmed it works - if you want to test it, that’d be most appreciated! https://t.cmpl.cc/qutebrowser.dmg

                                    1. 1

                                      App worked perfectly. Was able to download it and fire it up no problem. I’ll play around with it a bit more and let you know if anyone comes up.

                                      Sent from my qutebrowser

                                      1. 1

                                        Awesome, thanks for testing! I assume you’re on OS X 10.11 (~~Yosemite~~El Capitan)? I’d be really curious if it works on 10.10/10.9 as well.

                                        1. 1

                                          El Capitan actually (10.11.4 (15E65)). I think I might have an older machine I can try it out on, i’ll have to get back to you on that. I did receive a crash signing into github and reported it through the reporting dialogue box that came up. Not sure how that reporting system works and if you’ll eventually get the crash report but if there’s a better place for me to send it to you let me know.

                                          1. 1

                                            Hmm, I think you’re running into this Qt bug. I fixed it in Qt, but maybe for some reason the Mac I’m building the dmg on didn’t have the fix backported…

                                            I think you get an OS X crash report window? Can you look at the details there and confirm the stacktrace mentions WebCore::SocketStreamHandle::platformClose() too?

                                            1. 1

                                              Sorry for the delay, yes I see that line in the stack trace

                                              0   libsystem_kernel.dylib          0x00007fff9d1948ea __kill + 10
                                              1   libsystem_platform.dylib        0x00007fff8c61852a _sigtramp + 26
                                              2   ???                             000000000000000000 0 + 0
                                              3   QtWebKit                        0x000000010774bf04 WebCore::SocketStreamHandle::platformClose() + 84
                                              4   QtWebKit                        0x000000010774a79a WebCore::SocketStreamHandleBase::disconnect() + 26
                                              5   QtWebKit                        0x000000010773bf96 WebCore::WebSocketChannel::fail(WTF::String const&) + 710
                                              6   QtWebKit                        0x0000000107739375 WebCore::WebSocket::close(int, WTF::String const&, int&) + 325
                                              7   QtWebKit                        0x00000001077fb42d WebCore::jsWebSocketPrototypeFunctionClose(JSC::ExecState*) + 205
                                              
                                              1. 1

                                                That’s indeed the crash I suspected it was - I installed the patched Qt on my build machine and repacked, can you please try https://t.cmpl.cc/qutebrowser-dmgv2.dmg ?

                                                1. 1

                                                  Yup looks like that fixed the crash. I was able to sign into github no problem.

                                          2. 1

                                            Hi, it seems to run fine on 10.10.4 (Yosemite) for me. Good luck!

                                  2. 1

                                    I’ve been a heavy user of Vimium for a few years and I just tried this on my windows machine. It works really well! Will see if I can contribute to development, PyQt5 looks awesome to use.

                                    1. 1

                                      I’d be glad! Let me know if you need help :)

                                  1. 6

                                    It warms the cockles of my heart to learn that mutt’s still being actively developed :)

                                    I know I should migrate off of Gmail, but I have the keystrokes in muscle memory at this point.

                                    1. 2

                                      I have given up on trying to migrate off Gmail at this point. I’ve tried mutt, pine, thunderbird, and a host of other GUI based email clients (most of which are now defunct at this point, I’m looking at you sparrow) and none of them have stuck. The only caveat is offline email access is not all that important to me and I can using something like offlineimap to create a backup of the emails.

                                      1. 2

                                        Ah, yeah, Sparrow getting eaten by The Goog was super painful. I loved that client!

                                        1. 2

                                          How about FastMail?

                                          1. 1

                                            I recently just moved off of FastMail back to Gmail after using it for 1 year. I liked FastMail but at the end of the day I was missing the integration between the Google products and frankly the mobile app left some to be desired compared to the Gmail app, the web ui was fine however.

                                            1. 4

                                              Not to be too sardonic: but you mean the integration where people you e-mail with suggested in Google+, etc.? Or where images that are shared in an application covered by Google Apps for Work (Hangouts) show up in ad-mined services such as Google+ or Google Photos?

                                              One thing I have appreciated when moving away from Google Apps is that my data is much better compartmentalized and I decide what can be linked when.

                                        2. 2

                                          but I have the keystrokes in muscle memory at this point.

                                          But you can add (nearly) the same keystrokes to mutt as well.

                                          macro index y "<save-message>=Archive<enter><enter>"
                                          macro index d "<save-message>=Trash<enter><enter>"
                                          macro index * "<copy-message>=starred<enter><enter>"
                                          macro index,pager gi "<change-vfolder>inbox<enter>" "go to the inbox"
                                          bind index,pager a group-reply
                                          

                                          etc.

                                        1. 10

                                          It actually likely cost much more than $336k - possibly more than a million. There are a few parts at play here.

                                          There is a minimum obligation of $336,413.59 (box 26), likely to cover the base period of the contract. But, that’s just the first six months of a two year contract. The contract actually includes three more six-month option periods and has a total cost ceiling of a whopping $1,176,280.72 (see the supply the schedule.)

                                          There isn’t enough material to decide if the 18-months of option periods were actually funded. However, contractors almost always get these. Also, and this is a time and materials contract, so the TSA may have spent much less than $336k (you would need to look at invoices to see how much IBM actually billed.)

                                          Is $1.2M outrageous? The GSA contract vehicle (GS-35F-4984H) given is for general IT hardware, software, and services. The randomize contract is written for “mobile application development,” which means it was a services contract and mostly went to developers, engineers, project managers, etc.

                                          IBM has public rates available for this contract for 2016. Who knows what labor categories they used for billing the government, but going with a rate of $200/hr we get a maximum of about 5881 hours or about 3 person-years of effort.

                                          1. 2

                                            Yeah, and if you’re looking for the hourly rates from the government, https://news.slashdot.org/story/15/10/22/2336220/government-team-experiments-with-paying-for-small-open-source-tasks indicates that an average winning rate for a “Senior Consultant” w/ a BS degree and 5 to 10 years of experience is 171 USD/hour, which has to cover business expenses, overhead, supervision, contract searching, bench time etc. Compare that with the averaged salaried employee only making 50 USD/hour.

                                            1. 1

                                              According to this article[1] the app itself cost $47K and was only part of the entire contract.

                                              The total development cost for the randomizer app was $47,400, a TSA spokesperson told Mashable, which was part of the $336,413.59 contract. The spokesperson declined to elaborate on what else the contract entailed.

                                              [1] http://mashable.com/2016/04/04/tsa-ibm-randomizer-app/#1x4kszSOHPqo

                                              1. 1

                                                Thanks for clarifying; I’ve updated the post to include a lot of the info here, and linked to this comment.

                                              1. 16

                                                I am in the process of trying out emacs + evil after being a long time vim user. I started with spacemacs but quickly found it to be pretty confusing to setup and switched to just straight emacs + evil and haven’t had any issues. I have the same feeling for all the vim starter kits as well, they do too much and people don’t understand whats going on. I find immense value in setting up my environment from scratch and learning about the different pieces and how they work together. Yes spacemacs adds layers and some other configuration on top but I found it to be way more heavy handed and confusing than I needed.

                                                1. 7

                                                  I created the original Starter Kit for Emacs, and I fully agree. Back in the day (before the package manager) it sorta made sense, but these days the effort would is much better spent on creating and documenting individual packages that do one thing well. The Emacs Starter Kit is now fully deprecated, and the readme is just a document explaining why it was a bad idea.

                                                  1. 5

                                                    I’m also fine with emacs+evil so far. One hangup is that evil-mode interacts badly with some other packages and modes, though. For example, I use mu4e to read mail, and evil-mode breaks its main menu. There are usually workarounds, but if you use a lot of those modes together, an all-in-one setup where someone has already done the configuration to get everything working together might save time and hassle.

                                                    1. 2

                                                      Yeah thats a good point, to be fair I haven’t gotten far enough into spacemacs or emacs for that matter to experience many weird interactions. I have seem some weird behavior between evil and helm (I think) and some other modes (opening git interactive rebase seems to completely disable evil mode). I was going to give emacs+evil a few weeks and re-evaluate. If I end up switching back I will miss https://github.com/johanvts/emacs-fireplace though :)

                                                    2. 4

                                                      The most useful thing is that the layers provide consistent evil bindings. They also deal with a lot the quirks when integrating evil modes into holy things. Recreating that would be a lot of work.

                                                      1. 1

                                                        Yeah I’ve definitely noticed some of those quirks and don’t have a great way to figure out what they are and how to fix them. That’s definitely where spacemacs would come in but honestly it feels like an uphill battle of whack-a-mole.

                                                      2. 1

                                                        I had the same experience.

                                                      1. 2

                                                        My team, despite being mostly local, uses a tool called iDoneThis. We have an in-person/virtual standup Monday morning to sync on work for the week and then use iDoneThis to stay in sync for the rest of the week. This is really helpful for when people are out and want to stay caught up on what everyone is working on. It also really helps look back and reflect on what you’ve worked on.

                                                        1. 3

                                                          We played with iDoneThis at FreeAgent many years ago, and someone also pointed me at: StandUpMail which from the look of things is fairly similar. I can definitely see those being useful for us, so we might have another explore of those tools.

                                                          1. 1

                                                            I looked at iDoneThis a few years back for a very small team I led. I loved the idea and was shocked there was no open source clone. Ended up not using it because of size, but if I ran a team that was either bigger or distributed, a similar tool would be very helpful.

                                                            The question then becomes, when do you have the face to face between team members (managers will get their one to ones, of course). The options I see are:

                                                            • adhoc, just through the course of working together
                                                            • scheduled (I have heard of peer one to ones, which is an interesting idea)
                                                            • weekly group video calls, as other comments have mentioned