-
Last week was a sprint to finish stats assignments for end of term at the London School of Hygiene & Tropical Medicine:
- For Analysis of Hierarchical and Other Dependent Data, an analysis of how patients’ CD4 count trajectories varied depending on drug and three variables collected at baseline, using a mixed effects model and generalized estimating equations,
- For Bayesian Statistics, a ficticious analysis combining data from a randomized controlled trial, a case-control study, and a cross-sectional study – it’s a bit hard to believe, but even though they study completely different probabilities, the odds ratio of outcome versus exposure (or equivalent ratio of exposure versus outcome!) is a common parameter.[1]
- For Survival Analysis, Kaplan-Meier and Cox regression analysis for differences in time to death for patients diagnosed with Follicular Cell Lymphoma, by age, stage at diagnosis, and hemoglobin level (g/l) in a ~30 year cohort study, plus analysis of competing risks of relapse or death not from relapse.
This week is:
- figuring out if the data’s going to be ready for my summer project, and lining up contingencies if not,
- job marketing & networking, both in public health and back into the London software scene,
- as the English call studying, “revision” of the past two terms of coursework. Comprehensive exams are in early June.
[1] If you’re really interested, the probabilities in question are:
- For RCT, we administer an exposure, count outcomes, and then estimate probability of an outcome, given an exposure –
Pr(outcome | exposure). So in a non-inferiority trial, we’d measure ,Pr(Death|Drug A)vs.Pr(Death|Drug B). - For case-control, we collect some number of each outcome (called “cases” and “controls”), count exposures within each group (cases or controls), and then estimate probability of an exposure, given an outcome. So, again in this example,
Pr(Death|Drug A)vs.Pr(Death|Drug B). (Usually, case control is about some non-administered exposure…but this was an exercise.) - For cross-sectional, we collect some total number of people, count how many had every combination of factors, and then estimate the joint probability
Pr(Outcome AND Exposure)orPr(Outcome)|Pr(Exposure). So, again here,Pr(Death)Pr(Drug A), etc.
-
There are various ORMs/mappers, but most advice you’ll find (and what I’ll say, also) is to not use them. Wrap a database connection in a struct that has methods to do what you want. Something I’ve found conditionally useful are generators to write the function to populate a struct from a database row.
The community will also say to not use web frameworks, and again I’d agree. The stdlib http package provides a stable foundation for what you want to do. You’ll have more luck looking for packages that do what specific thing you want, rather than thinking in terms of frameworks.
All that said, some coworkers like echo but I can’t for the life of me understand why. Any web-oriented package shouldn’t need to give a shit if it’s hooked up to a tty or not.
-
The problem is that when you do search and filtering on various conditions (like in a shop) you don’t want to resort to sql string stitching, I wasn’t able to find anything nice when I looked at the docs, for example in gorm:
db.Where("name = ? AND age >= ?", "jinzhu", "22")- I expect that when you have 20 conditions with different operators and data types you will end up having a bad time.-
-
sqlx
https://godoc.org/github.com/jmoiron/sqlx#In is pretty useful when you need it.
-
-
We built something in house that is very similar in spirit to sqlx but adds a bunch of helpers.
-
-
https://github.com/Masterminds/squirrel (which kallax uses) seems somewhat akin to the SQLAlchemy expression API. (And yeah, to me, that’s a great part of SQLAlchemy; I’ve hardly used its ORM in comparison.)
I went from Python + heavy use of the SQLAlchemy expression API to Go and got by OK with just stdlib, but part of that was that the work in Go had far less complicated queries most of the time. So, not the best comparison maybe.
-
I support the advice to not use mappers like ORMs, but I also agree with what you said. The middle ground seems to be query builders.
If you use Postgres as your DBMS by any chance, I advise you to make sure that the query abstraction layer of your choice doesn’t do query parameterization by string interleaving but utilizes PQexecParams underneath instead.
-
I haven’t used it but I think Dropbox built a library for that. https://godoc.org/github.com/dropbox/godropbox/database/sqlbuilder
-
-
-
- End-of-module assignment for Generalized Linear Models.
- End-of-module assignment for Statistical Methods in Epidemiology.
- Proposal for working on TB surveillance data as the project for my MSc in Medical Statistics at LSHTM.
-
The LSHTM MSc Med Stats inference assignment went out today, so mostly that.
Otherwise, this week is inference, regression, more epi, more clinical trials, more intro bayes.
Oh, and the poor artifex.org server may finally have working spam again, now that it’s back in a colo of sorts.
-
Well, I thought it actually set out to offer an argument – it does seem valid to say he picked only facts in support of his case (and spent limited time considering alternative explanations, too?).
But, I agree it’s probably not the best way to do so.
If you didn’t read it already, these comments from women at Google are wortth your time. They’re a better starting point for the debate. (It helps that they haven’t chosen to write in the format of “here’s what we’d say to James Damore after firing him”)
https://blog.ycombinator.com/ask-a-female-engineer-thoughts-on-the-google-memo/
-
friendlysock
edited
1 year ago
|
link
| on:
Palantir: the ‘special ops’ tech giant that wields as much real-world power as Google
This sort of article isn’t a good fit here–maybe try HN?
There’s no technical content, nothing really educational that would help in the daily practice of development, and the article itself straddles the queer line of being both a vague “these are evil spooooky intelligence company” and being nearly free advertising for Palantir.
Please don’t submit this stuff.
-
There’s no technical content
This article is all about the negative applications of modern day technology and how it’s used. If anything we need more of these sorts of articles. How else can we have discussions about the ethics of the applications of technology?
If you don’t want to have such discussions, no one is forcing you to participate in them. There’s a “hide” button for exactly that reason.
-
If you don’t want to have such discussions, no one is forcing you to participate in them. There’s a “hide” button for exactly that reason.
That can justify posting anything to lobste.rs. Don’t like posts on My Little Pony? Just hide them since nobody is forcing you to participate. That does not mean any content is correct for lobste.rs.
This particular question of content related to moral/ethical usage of technology has come up multiple times, with you in particular. I believe @angersock’s perspective is that purely discussing the moral and ethical usage of technology is not content appropriate for lobste.rs, while your perspective is that it is. I am not sure how to resolve this, to be honest, but I think it’s worth acknowledging the source of disagreement.
In particular:
How else can we have discussions about the ethics of the applications of technology?
Just because something is worth discussing does not mean lobste.rs is the correct venue for it. I have not seen any evidence that people on lobste.rs don’t think the moral/ethical usage of technology is an interesting and important topic, just that they believe lobste.rs is a community that should focus more heavily on the technical part rather than the morals.
-
A couple of people keep repeating this self-enforced angersock rule like gospel, but I don’t understand where it came from. It looks increasingly like an arbitrary personal preference by a vocal minority. Technology has always had ethical implications and these should be discussed. If not here, where else? I am certainly not going to go to HN for that…
When I link to articles on fairphone’s blog is that also off-topic? They are usually touching on ethical issues, yet nobody shouts at me for submitting such links. Same for many talks from conferences like CCC.
-
I am certainly not going to go to HN for that…
Lobste.rs started as a technically focused community. If enough fluff pieces get the same standing here as on HN, the discussion here will be just like on HN. Incidentally, unlike many here, I don’t have a problem with HN discussion quality, but one should realize that topics you discuss define the audience that will end up participating.
-
A couple of people keep repeating the self-reinforcing “but but but we need to talk about the morality, man!” argument, but I don’t understand where it comes from, etc.
I’ll try to make this painfully clear for anybody who hasn’t already taken a side:
Think pieces about technology and culture and ethics are everywhere on the internet. There are entire industries whose sole business is muckraking and masturbating about technology. Some forums regularly navel-gaze on these things. It’s not like we can’t get that content elsewhere.
Discussions about ethics and politics fall very quickly towards relying on and normalizing news and anecdata. Neither of these tend to encourage good discussion, since it ends up with people either complaining about the inaccuracy of news (or, worse, taking it at face value) or slapfighting over whose anecdote is most anecdotal. This rarely results in things that help improve our abilities as practitioners of engineering or software.
The people who demand politics and “ethics” tend to correlate highly with people willfully ignorant of or outright hostile to the people with whom they have policy differences. These people cannot and will not agree to disagree, they will petition mods behind closed doors to remove what they view as “problematic” users, they will not argue in good faith without resorting to nebulous arguments, they will not acknowledge science or statistics that raise inconvenient questions. They will attempt to blacklist users with whom they disagree, they will use conversations out of context via screencap in order to summon mobs, they will reflexively downvote even reasonable submissions and comments because they view the posters as stupid conservatives (or even subhuman, but their knowledge of history has a blind spot here), or whatever else. And if challenged on this, they are obstinate and self-righteous and just generally quite unpleasant.
(And lest you get the wrong idea, this problem is not limited to one side. There are folks that spiral off into long tirades about the evil progressives, the massive socjus conspiracy, and all sorts of other garbage. Culling one group of users or the other is not going to fix it.)
That’s all fine, and we as a community can happily include such delightful people, if we stick to straightforward discussions about technology. There are users whose politics I do not share but whose expertise I am quite happy to learn from. I am happy to overlook that in favor of learning and civil discussion. The second we veer into policy (“because we need to raise awareness”, “because what about the consequences maaaan”, etc.) we are forced to deal with all of messy bullshit of politics and ethics and partisanship and privilege and the whole damned thing. And it is never something that goes well. It never enriches our lives as users. It brings out the worst in us.
Finally, none of these arguments matter. Statistically, a reader or thread member here is unlikely to be working at the Open AI foundation. They are unlikely to be working in a role where they could, say, change the exploitative behavior of GOOG or FB. They are unlikely to be willing to change jobs or forgo paying rent to stand up for their beliefs in any meaningful gesture. Most of the big discussions on ethics become moot once you accept that capitalism and the free market are going to enable consumers to do whatever seems like a good idea–and if you don’t accept that, at least partially, you probably don’t have a computer or network connection to post here and gripe about it.
We’re generally quiet and polite technologists, happy with and supported by the status quo, so let’s not go cluttering up the place and introducing unpleasantness in our desire to pretend otherwise.
-
It’s not like we can’t get that content elsewhere.
It actually is the case that we can’t get that content elsewhere.
The most important audience for this content is those who work at these companies, many of whom visit websites like Lobsters, Hacker News, Slashdot, etc. And on most of these websites, where it is most important to have this discussion, apologists such as yourself appear out of nowhere to police the conversation and shut it down.
-
HN and Slashdot and Reddit frequently have conversations about this sort of stuff, and do so without getting shut down (at least if you follow the hivemind on which side you talk about).
It’s disingenuous to act like that is not the case. It’s also disingenuous to act like other sources, like NPR, TED, BBC, New Yorker, and a myriad of other sources are a) not covering this material thoroughly and b) users here are not informed by those or other sources.
I appreciate that your activism needs an audience where it will be most effective and the relative refuge of lobsters for technical folks makes it an attractive target, but please do not be surprised or mystified that this kind of disruption isn’t being embraced unquestioningly.
If you feel that there really and truly isn’t a place doing this meaningful discussion, by all means, make one.
-
It bemuses me to no end that the people who use the word “politics” to shut down this conversation (such as yourself) are exactly the ones who turn the conversation into a game of politics in the first place.
Look at the comments you and @apy have written, and compare them for substance to other comments in this thread, like for example this one by @mordae.
A world of difference.
It’s also disingenuous to act like other sources, like NPR, TED, BBC, New Yorker, […]
Since this entire conversation you’ve started is off-topic, I will pick only NPR as an example, but the same applies to every other source you’ve quoted (especially TED): I distinctly remember listening to NPR’s coverage of the Senate Torture Report while in my car, and I remember NPR brushing the entire thing aside as if it were no big deal. NPR is part of the MSM, and the MSM is a single entity (literally). The critiques of government in it are absurdly mild because the MSM is the government’s mouthpiece.
So: kindly stop being an apologist for atrocities everywhere you go, and kindly stop derailing the entire conversation into off-topic bullshit land.
-
So: kindly stop being an apologist for atrocities everywhere you go
Is that really a fair characterization of @angersock’s comments?
-
Is that really a fair characterization of @angersock’s comments?
Yes. Previous examples include being an apologist for the 9/11 attacks and for drone bombings of civilians.
-
I’m unsure how those examples are of apologism of those things, since in those examples you provide I was pretty explicitly sketching out how things were off-topic (and not defending the things themselves, though apparently your bar for “supporting” a thing is “failing to loudly decry as terrible as often and as loudly as possible everywhere one can”.
Anyways, I thought you weren’t going to spend one more second on this?
-
I’m unsure how those examples are of apologism of those things, since in of those examples you provide I was pretty explicitly sketching out how things were off-topic
Recent academic research is not off-topic. The impact of technology on society and its relevance to the
security,privacy,culture, andlawtags are also not off-topic.But all of your comments so far (like this one here) have been either off-topic or trollish.
Your job appears to be to prevent people from thinking about these subjects or discussing them under the pretense of policing on-topic content as being off-topic.
Therefore you are an apologist for atrocities, although admittedly a very good one.
Anyways, I thought you weren’t going to spend one more second on this?
And for about a year I made good on that. It’s interesting to see that you’re still here, still thought-policing Lobsters as usual.
-
I did a quick dictionary search for “apologist” because your usage doesn’t seem to match my understanding. The definition I found is:
one who speaks or writes in defense of someone or something
The links to comments by @angersock you posted do not appear to defend anything but rather argue that they are not appropriate for lobste.rs. Whether or not content is appropriate for lobste.rs is a valid discussion, IMO, but muddying the waters by claiming that makes someone an apologist seems an unfair accusation.
I believe it would be correct to say @angersock is an apologist if he, for instance, said that drone strikes were a good thing, and <here’s why>. @angersock even goes as far as to say:
This is a political and policy article and as outrageous as it is it doesn’t have anything to do with technology.
I interpret “outrageous” here to mean that if the contents of the article are true he finds them loathsome but that doesn’t mean the content is appropriate for here.
-
I interpret “outrageous” here to mean that if the contents of the article are true he finds them loathsome but that doesn’t mean the content is appropriate for here.
I look at people’s actions, and the consequences of those actions, more than I do at their words.
The consequences of shutting down all on-topic discussion about the negative impacts of technology, as well as new research that reveals, through technology, important facts about our reality, is that those negative aspects of reality not only continue, but grow.
It is in that sense that he is an apologist. Just as the definition says, he defends atrocities by thought-policing all discussion about them.
-
-
Why can’t the disagreement be about about if something is on-topic or not rather than the other side being an actively bad person?
It can, and it was. That conversation ended up devolving into nowhere. A clear pattern has emerged demonstrating a lack of good faith. I have no interest in repeating the same exercise.
-
-
-
-
-
-
-
-
-
-
-
-
Given the long comment thread and no “parent” button in lobste.rs, I can’t tell if “You” is directed at me or someone else. But assuming it’s directed at me:
I don’t know if those links you have posted are off-topic or not, I don’t read every link on lobste.rs.
My comment, also, was not about defining what is off-topic or not and it, especially, was not about defining on-topic content in terms of links I like. My comment was actually the opposite and about framing the discussion in terms of what lobste.rs considers on and off-topic. And my attempt also seems to have been a failure.
To change your question to “I don’t understand why you can’t just ignore links you consider off-topic”. The reason one cannot do that (or at least @angersock can’t, if I can try to speak for him) is because one perspective as that the on/off topic filter is supposed to happen at the point of submission, not the point of reading, in order to keep the total content of lobste.rs high quality, where high quality includes “on-topic”. This perspective is that lobste.rs is not meant to service all the needs of the readers but a specific niche of technology.
-
-
-
-
This particular question of content related to moral/ethical usage of technology has come up multiple times, with you in particular. I believe @angersock’s perspective is that purely discussing the moral and ethical usage of technology is not content appropriate for lobste.rs, while your perspective is that it is. I am not sure how to resolve this, to be honest, but I think it’s worth acknowledging the source of disagreement.
I’m frankly getting tired of this debate. I am one of the proponents that sees this as fit for lobste.rs and a sizable group agrees.
If people think this article is bad, they should write about it (and angersock did), but keep the framing of “is this appropriate here?” (or worse: “this isn’t HN!”) to themselves.
Seeing that every of these articles has at least one sub-thread rehashing all this is really annoying.
-
If people think this article is bad, they should write about it (and angersock did), but keep the framing of “is this appropriate here?” (or worse: “this isn’t HN!”) to themselves.
To be blunt, no I will not keep this to myself. The reason this keeps cropping up is we don’t want this site to turn into yet another “technical news/etc…” site devolved into yet another, digg, reddit, slashdot, hacker news, etc…
If you look at the scoring, its split down the middle so I do not believe your viewpoint is the one that should prevail and I will mark them down as off topic every time. All I want is ONE site to not devolve into pointless political debate and all that entails and attracts. One can’t escape politics, but we can at least confine it.
-
I will mark them down as off topic every time.
Please do not downvote other users just because they have a viewpoint you disagree with–only use the
off-topicflag if they are truly off-topic in a thread. -
-
One can’t escape politics, but we can at least confine it.
Or discuss it. I enjoy the political debates here a little more than at other places, and I absolutely don’t see politics as a bad thing - I think it isn’t debated enough in technical communities.
I had rather preferred a good dissection why this is a bad article (and @angersocks initial post does kick that off), without the usual meta-debate.
-
If we have to go down this route, then make a politics tag I can ignore.
The thing is I agree with everything you’re saying regarding politics, but disagree in kind that we need yet another technical focused site to yet again start hashing out politics. I feel we have enough sites that people keep leaving in large part due to the whole eternal september effect.
Essentially, I’d rather not have lobsters go down the same well trod road of political discussion which seems to just breed a very insidious and nasty form of online “debate” that in the end just causes people to shut down and stop contributing.
-
-
-
If you have an idea for how to come to a conclusion on these then please contribute it. Given it’s one of the more polarizing things in lobste.rs I’ve seen, I doubt it will go away on its own any time soon.
but keep the framing of “is this appropriate here?” (or worse: “this isn’t HN!”) to themselves.
This means that all the content is appropriate for here, though, which kind of defeats the purpose? If we all keep the thing the disagreement is about to ourselves then we won’t have disagreements. Sure.
-
-
That can justify posting anything to lobste.rs. Don’t like posts on My Little Pony?
That’s a ridiculous strawman, having exactly zero to do with the argument being made.
My Little Pony has no technical content, whereas discussions about privacy and security and how technology affects them (e.g. snowden leaks, etc.) are completely appropriate for this site and make regular appearances.
-
-
Ethics in technology are important of course. But online discussions about ethics in technology are almost always tedious and useless.
People make noise about Google/Uber/Palantir etc being shitty companies. People agree with each other that they are shitty companies. Nobody employed by those companies quits their job or lifts a finger because they like making $250k+ writing piles of JavaScript that don’t work. Repeat next week.
If we could have discussions about ethics that went beyond that then I would welcome them, but I expect (like everywhere else on the internet) we would have the same simple conversation over and over and it gets quite boring.
-
And yet, it seems like the alternatives to discussing ethics online aren’t great, either. Discussing ethics with one’s acquaintances only does so much to better the world, for instance.
Is there some other means of inquiry into morals that you, as an engineer (or just a lobste.rs reader) would find more interesting?
-
-
-
-
But: wouldn’t a more thoughtful piece on the metastasization of software platforms be really interesting? I’ve often thought of this headline in the years since it came out but always forget what the article has to say itself. It seems like another line of inquiry, though still contentious, may have more to say.
For instance, some languages/frameworks gather more small, hard-to-keep-up dependencies than others.
And, I’m obliged to agree with an old colleague, that maintaining an event-driven application gets much harder as the team grows, compared to one where the runtime schedules threads/coroutines/etc. for you.
-
Ah, yes. A few of us built a product for this at Newfield Wireless starting in 2003. Even back then, we could get ~100m resolution pretty easily.
It still sells, I think.
-
“How monotonously alike all the great tyrants and conquerors have been; how gloriously different are the saints.” - C.S. Lewis
One reason I think this resonates is our industry feels more in the throes of extractive, me-me-me culture than ever before, mostly due to the bubble. Tech has been subsumed by the larger business culture, which sees integrity is a liability. I think most people don’t actually share this view, but feel they have to acquiesce to it (be it conscious or not) in order to survive. In effect, the culture is polluted by self-aggrandizement. To see a profession of values higher than one’s own personal success is refreshing.
-
Berkshire’s a notable exception – the biennial letter from Warren Buffett to mangers, for some years running, says:
This is my biennial letter to reemphasize Berkshire’s top priority and to get your help on succession planning (yours, not mine!).
The priority is that all of us continue to zealously guard Berkshire’s reputation. We can’t be perfect but we can try to be. As I’ve said in these memos for more than 25 years: “We can afford to lose money – even a lot of money. But we can’t afford to lose reputation – even a shred of reputation.” We must continue to measure every act against not only what is legal but also what we would be happy to have written about on the front page of a national newspaper in an article written by an unfriendly but intelligent reporter.
(https://www2.bc.edu/robert-radin/Administration/Buffett%20Managers%20Letter%202008.pdf , http://www.berkshirehathaway.com/letters/2010ltr.pdf)
-
-
At home, just trying to prepare a VM image for use on this decade’s deployment of Artifex.org. (Nothing more that dovecot + postfix + ssh + nginx, but I took an excursion to automate bringing up single instance K8S without minikube. Was that a good idea?…)
Oh, and still working my way through The Origins of Totalitarianism! I gave up on a recall battle with a fellow London Library patron and bought my own copy….it’s macabre but unsurprising that Penguin has made a new, 2017 edition.
Otherwise, I need to circulate some more in the London technology scene. It’s been a different world than all those years in the Golden State.
-
This guy sounds like a real treat to work with.
-
-
A few notes from the office:
- We use roles and grants quite extensively in our main PostgreSQL DB, and for at least 5 years or more. To some extent this has been a good thing. It’s also meant a lot of complications. The phrase “update the grants file” connotes an unpleasant and somewhat terrifying task, for instance. (And yet, update it we must.)
- We’ve tended to use views instead of column level security.
- Row level security in PostgreSQL seems to be pretty performance intensive. But, mostly we just haven’t needed it yet.
Roles are also proving fairly useful for a data warehouse DB (which FDWs to other DBs, btw).
-
Is it a proposed feature, or is it in mainline? Is it micro-kernel features tacked onto Linux? How much is the overlap with current primitives?
-
More gifts from the greeks: https://www.phoronix.com/scan.php?page=news_item&px=BUS1-In-Development
-
-
I’m a CloudFlare engineer and came across this post last night as well.
Both of the issues that slashgeek raises (POP connectivity and unfair CAPTCHAs) are ones that we are always working to improve, both for our customers and for internet users regardless of their nationality. It’s no good that the service fell short for slashgeek and the users of his ISP.
CloudFlare’s position has always been that it should make the internet work better for everyone. If we’ve failed to do so here, then we’ll have work to do to remediate it. I’ve pointed our communications team to the thread so they can follow up on this.
-
This is fair, I’d rather be re-reading the original text.
Before “five whys” was coined, it was just a section in Taiichi Ohno’s Toyota Production System, and a pretty good one (https://books.google.com/books?id=7_-67SshOy8C&lpg=PP1&dq=toyota%20production%20system&pg=PA17#v=onepage&q&f=false).
Alas, but, making these few paragraphs into a practice, and then making that practice into a marketable trend, has indeed gotten us away from what Taiichi Ohno was trying to get people to do.
-
It’s good to see some of your work out in the open, @prideout! Godspeed the mappable.
-
For me, this is actually a kind of interesting piece on pedagogy (that would be a tag). I just took a class in the Berkeley Math Department this spring while working full time as an engineer in West Berkeley (it’s a different class, but the professor in the underpopulated lecture photo was, indeed, my professor). I’d tend to agree with the author about how things get taught.
For people who do find this topic of interest, another piece that did come up some time ago was http://bentilly.blogspot.com/2009/09/teaching-linear-algebra.html
-
Work: yet more ontology matching. So far, it’s looking like Kinesis will serve our needs for the data pipeline part of it.
Home: yet more log parsing with Heka. Not very interesting, but almost done.
-
Work: more ontology matching and plumbing, at least once updates to the mobile client build / deploy pipeline are done.
Elsewhere: a few more heka + lpeg log parsers for the Artifex server.
Comprehensive exams this week, then interview prep for the following week.
Paper I, Wednesday:
Paper 2, Friday: some of the above, plus GLM X 2, Survival analysis X 2, and Bayes.