Its really easy to write a simple HTTP server. I’ve done it a couple of times for weird languages like Self. You can do it in an afternoon and you end up with something which is fast enough and usable enough for simple websites with low traffic and all you need is to be able to talk to your OS to open a socket, read, write and close it.
There is no way that I’m going to write a HTTPS server. Which is a shame because it means that small language projects like Self will need to rely on a bunch of third party C code and third party infrastructure (letsencrypt, caddy, openssl etc) to serve a simple website.
Any HTTP Server can be easily converted into a HTTPS server by piping it through a SSL proxy. It’s the same protocol going over the pipe, after encryption there is no difference.
Sure, that’s what letsencrypt, caddy, openssl, etc provide you: a way to turn your simple HTTP server into a public facing HTTPS server. But the cost is that a small protocol which could be completely written in house for fun now needs a whole bunch of complicated C/Go/etc code and systems written and hosted by someone else…
At the risk of being overly reductive you’re already depending on a bunch of code someone else wrote unless your HTTP implementation also included the TCP and IP layers. Adding in TLS can be thought of as just inserting one more layer to the several that already exist beneath you.
(A complicated one you might need to configure and that isn’t provided by the OS, I grant you)
I was reading your post and wondering if the size difference between a standard kernel-space TCP implementation and openssl was negligible or not.
find linux/net/ipv*/tcp_* -name '*.c' -o -name '*.h' | xargs cat | wc -l
25119
find openssl/ssl openssl/crypto -name '*.c' -o -name '*.h' | xargs cat | wc -l
285101
Turns out t’s an 11.3 ratio, it is not negligible at all.
I was actually not expecting a difference that big!
[edit]: I just re-read my comment, do not interpret that as an attack, it is not :) You just itched my curiosity here!
On top of that, if we compromise performance for code size by dumping optional parts of the spec, we can get a minimal functional TCP stack in an amazingly small amount of code (cf uIP, lwIP, and the fabulous VPRI parser based approach in Appendix E of http://www.vpri.org/pdf/tr2007008_steps.pdf)
Come on now, there’s a big difference between depending on, say, BSD sockets and depending on an SSL proxy like nginx or something.
I’m more familiar with using languages/frameworks with built-in support. If you’re implementing it by a proxy that’s obviously a whole other component to look after.
A separate daemon adds more complexity (and therefore fragility) to the system. BSD sockets are well understood and aren’t something the sysadmin has to manually set up and care for.
To divide by 3, I used a multiplication trick. On tiny devices, division is often implemented as a (slow) library call, but if you’re dividing by a constant, you can usually find a good equivalent fixed-point constant. “One third” in binary is “0.5555…” repeating, so using a 16-bit fixed-point fraction, you can multiply by 0x5556 and shift right 16 bits.
Why 0x5556 and not 0x5555?
Because shifting right by 16 is the math equivalent of “floor” instead of “round”, so we need the result to be not just “as close to 1/3 as possible”, but the error needs to be on the high side instead of the low side.
0x5555 * 3 = 0xffff – shifts right to zero
0x5556 * 3 = 0x10002 – shifts right to one
I’ll add this to the post, thanks!
Because the number 0.555555… is a little closer to 0.5556 than 0.5555. When they say “in binary” I think they mean “in hex” too.
0x0.55555… is closer to 0x0.5555 than 0x0.5556. Remember, 0x0.5 is five sixteenths, significantly less than a half (which would be 0x0.8).
import java.util.stream.*;
public class Triangle {
public static String triangle(int rows) {
return IntStream
.rangeClosed(1, rows)
.boxed()
.map((row) -> Stream.iterate(row % 2, (n) -> n == 0 ? 1 : 0)
.map(Object::toString)
.limit(row)
.collect(Collectors.joining(" ")))
.collect(Collectors.joining("\n"));
}
public static void main(String[] args) {
System.out.println(triangle(4));
}
}
Yes, it looks much uglier than Haskell version, but constructs are almost the same.
(I’m not arguing that you can avoid imperativeness in Java, you almost can’t, and even Streams are imperative but somewhat composable)
to be fair, java 8 (and streams) had only been out in the wild for 5 months when this was written.
But are those features of Java actually taught at university? Because they sure don’t spend time on them where I’m at.
I haven’t run across it, though I could imagine some contexts where they might be. I might even consider doing it in the future, given the right circumstances. But I think they’re not that likely circumstances. Usually university courses are trying to teach at least two things with a programming language (or language construct): the language itself, and some broader principle, which students will hopefully retain long-term and be able to apply even when languages inevitably change over. So typically Java has been used either as “generic intro-level language” or “OO language”, which Java 8+ streams don’t fit that well into.
In universities where Java’s used as the standard intro-level language, there’s some advantage to continuing to scaffold atop it for further things, because you can assume everyone already knows the basics. But it’s falling out of favor for intro-level classes in favor of Python. So I think you will either see people introducing this “streams” style of programming in Python if the circumstances fit, or else they’ll stick with a more traditional introduction to sequence filtering/mapping/etc. in the context of an introducion to functional programming, in Haskell or ML or maybe some Lisp dialect.
So I think you will either see people introducing this “streams” style of programming in Python if the circumstances fit, or else they’ll stick with a more traditional introduction to sequence filtering/mapping/etc. in the context of an introducion to functional programming, in Haskell or ML or maybe some Lisp dialect.
I’m aware of some libraries that provide special syntax for that in Python, but I don’t think anything other than list comprehensions (and maybe higher-order functions) for this kind of stuff is considered “Pythonic”.
I did much the same but tried to introduce names in the same places as in the original; as relatively idiomatic Java that’s:
public class Triangles {
public static void main( String[] args ) {
System.out.println( triangle( 4 ) );
}
public static String triangle( int n ) {
return IntStream.rangeClosed( 1, n ).mapToObj( Triangles::line ).collect( Collectors.joining( "\n" ) );
}
public static String line( int n ) {
return alternating( n ).limit( n ).mapToObj( Integer::toString ).collect( Collectors.joining( " " ) );
}
public static IntStream alternating( int start ) {
return IntStream.iterate( start, i -> i + 1 ).map( i -> i % 2 );
}
}
You can also go totally crazy, but I wouldn’t recommend it:
public class Triangles {
public static void main( String[] args ) {
IntFunction<IntStream> alternating = i -> IntStream.iterate( i, n -> n + 1 ).map( n -> n % 2 );
IntFunction<String> lines = l -> alternating.apply( l ).limit( l ).mapToObj( Integer::toString ).collect( Collectors.joining( " " ) );
IntFunction<String> triangle = n -> IntStream.rangeClosed( 1, n ).mapToObj( Triangles::line ).collect( Collectors.joining( "\n" ) );
System.out.println( triangle.apply( 4 ) );
}
}
I wrote mine too! ;)
Funny how similar Rust and Haskell are in those small examples.
Plan 9 had its own assembler and the one in the Go toolchain is based on it, which might explain its use here. Like you I’m not sure why the prominent billing in the title of this post, the actual page doesn’t make a huge deal out of it.
IIRC, plan 9 assembly is platform independent; the plan 9 assembler takes it as input and outputs platform-specific machine code. This means it’s more portable, basically.
A single anecdote honestly doesn’t even hint very hard that this is a real thing. Confirmation bias is incredibly strong. In the absence of significantly more rigorously-collected evidence, I’m very strongly inclined to say the ad was relevant by pure coincidence.
Yea, there are many possibilities that are far more likely than the one they propose.
Simple coincidence is definitely one. Or someone else in the group did a search while connected to the same hotspot, or while advertisers had figured out they’re in the same household.
People often overestimate how random and unique their behavior is. You can often tell a lot about someone based on a few demographics…and can make surprising correlations.
It may be as simple as their age/location/profession to tell you they’re likely to consider buying a projector in the next 6 months. Perhaps they read an article about setting up a movie night, or gaming outdoors, etc.
Oh, I didn’t know. Thought it was a picture service. Only seen it occasionally, never used it.
Thanks for clearing it up!
It can record videos with sound. Microphone access is required to use the camera in the app as a result.
Interesting thing about this task is that I think static typing in a language like Java gets in the way more than it helps.
I haven’t used Java in a long time, but I would spend most of my time figuring out how to make a list that contained both integers and List objects. Maybe Oracle finally got around to making the primitive types real objects and it wouldn’t be so difficult now?
In Common Lisp it’s almost trivial, although this version isn’t as fast as the one in Alexandria:
(defun flatten (lst)
(if (listp lst)
(apply #'nconc (mapcar #'flatten lst))
(list lst)))
I’m guessing the “correct” way of doing this in Java would be to make a List<Object> and then fill it with Integers and other lists. Basically completely sidestepping the type system. This is unintuitive, and so you’re right that this question is probably a better fit for a dynamically typed language.
Alternately, in a language like ML, Haskell, or even Swift, you can use a sum type, where you can specify that the list contains objects that can either be other lists or integers, and have the type checker enforce that they can’t be anything else (unlike the Java example where you could theoretically stick any object in that list).
[Comment removed by author]
Since the article was complaining (okay, stating) that nobody used Java 8 I ended up implementing this solution: https://gist.github.com/calumleslie/8fe772225b8403e515286a56b965b985
Quite fun to do, but I’m not going to claim the support code is short. The solution is one (recursive) line though!
Edit: I was interested about how this actually executes, so here is the nightmare stack from the point where the system is evaluating the ‘5’ node.
Interesting thing about this task is that I think static typing in a language like Java gets in the way more than it helps.
That’s what makes it a strange interview question in this context imo, where it seems they’re hiring for a Java shop, interviewing candidates who responded to a job ad looking for Java programmers. Flattening a list is an introductory textbook example in dynamic languages where you can actually write literals like “[1,[2,3], [4, [5,6]]]”. But it’s not really a introductory Java textbook example.
It’s of course solvable in Java, and a broadly competent programmer should be able to come up with a solution in an hour. But I’m not sure it serves as a great fizzbuzz-type baseline filter when hiring for a Java shop. You probably end up filtering more on whether the candidate has previously encountered a flatten function in a different language and remembers the algorithm (maybe they went to a university that had a Scheme course), and/or whether they’re comfortable with doing ugly run-time type things in Java. Which are not useless skills, but there are probably lots of reasonably but narrowly competent Java programmers who would get hung up because it’s a bit of a “foreign” example to someone who works entirely in Java.
Pretty easy in Python too:
def flatten(lis):
ret = []
for item in lis:
if isinstance(item, int):
ret.append(item)
elif isinstance(item, list):
ret.extend(flatten(item))
return ret
Using generator and yield from would be my first choice, but yeah… What’s so hard about that? I mean… It’s literally a five minute task.
It’s only easy if you recognize the right case analysis. I use to TA for a programming languages course (mostly juniors and seniors), and one of the first assignments contains a question that asks you to write this function in a lisp-like language. Tons of students trip on it. A lot of them do get it right, and some of them write sub-optimally correct solutions (i.e., redundant case analysis). If you’re not use to thinking about recursive structure, then it can be tricky!
N.B. The assignment is given after a lecture that covers rose trees, so we weren’t just throwing them to the wolves. :-)
def flatten(source):
for item in source:
yield from flatten(item) if isinstance(item, list) else [item]
yield from ftw indeed!
I haven’t used Java in a long time, but I would spend most of my time figuring out how to make a list that contained both integers and List objects.
I think I would start by taking a step back to see if your data are being properly represented by objects. Java is, after all, OO. If you need a heterogeneous collection, it should be typed to store a parent class of the subclasses you want to store. Asking how to collect objects and lists is too abstract; rather ask how do I collect records about single family homes and also apartment buildings that are composed of multiple units.
its the blog post:
This happened not before trying to convince @Unknwon about giving write permissions to more people, among the community. He rightly considered Gogs his own creature and didn’t want to let it grow outside of him, thus a fork was necessary in order to set that code effectively free.
Looks like there was some minor drama in the Gogs world and that the primary developer of that project was away for a while. Since nobody could push changes they forked the project. An issue on the Gogs Github seems to indicate the protects have differing philosophies now.
Looks to me like your typical governance fork, like with ØMQ/nanoMSG, egcs/gcc, eglibc/glibc. I would bet on the fork with a more open community.
Here’s what happened with ØMQ (but in reverse, name-wise):
http://sealedabstract.com/rants/nanomsg-postmortem-and-other-stories/
People over code, the best is the enemy of the good, worse is better. Instead of being conservative, accept more input from the community, grow and breathe enthusiasm into the community. Any warts introduced by one contributor will be polished by another.
I think the DailyDot is incorrect, or at least confused. There was never, to my knowledge, a way to enable any sort of device (full or not) encryption for the eReaders. (Kindle Whitepaper, DX, etc) The Android-based tablets? Yeah, and as the screenshot shows, Amazon no longer supports it. The only encryption I know of on the eReaders is the DRM for the ebooks themselves, which I doubt Amazon is too keen on removing any time soon.
I can’t find a license for this code, which is disappointing given how prominently the code of conduct is displayed. As it is, there isn’t much here.
Yeah, I noobed out, sorry. I wasn’t expecting anyone to really really pay attention here until next week, when I had more content up and was gonna promote it for real. Oh well.
This looks like a project by Lobster’s steveklabnik
Yeah, it is. I skimmed it a few days ago and it only had a handful of sections…now it has a dozen. Definitely a work in progress, and mostly a hobby/teaching OS if I understand correctly, so I imagine a lot of stuff is still being fleshed out.
Neat stuff steveklabnik!
now it has a dozen.
I rode a bus home from my mom’s for 8 hours on Saturday, and got a bunch of drafts up… then got back to the internet, saw it was on HN, and said “well, I might as well publish the drafts, since people are checking it out and there isn’t a ton yet”.
So what’s there is rough but it will get better.
There’s an issue for this which is being worked on. Only complicated since it’s derived from another project.
Question about HTTP status codes in general: should they be represented as int or as string? It’s prolly inconsequential, though I’ve seen both choices floating around.
IIS will pass fractional sub-codes so I suppose it might be too support handling those?
Oh wow, hadn’t encountered that. Yes, strings would be helpful there. Thanks for passing that along.
I’d answer “no”. They should be represented as strongly typed instances, though probably with the option of giving a custom one - possibly enums with the correct ints would be adequate. Look at spray.io (or its spiritual successor http4s) for a library that I think handles status codes really well.
It’s appropriate that an HTTP library would define some response code types for the programmer. Yet it’s interesting to note that both of the libraries listed choose to represent the “raw” response code as int. Again, inconsequential, but curious…
I would use int only. Because range of the number also matters and int are easier to compare. Check these lines. For example, any code between 200 and 299, including those, is considered success.
Equivalently, for a string type, you could check the first character to determine which class the response falls in.
For him the scrolling becomes natural, comfortable and predictable.
I would have picked better examples. The intence page doesn’t feel natural or predictable at all. It has the wrong “intertia” on iPhone. Exactly what this library claims to get right, it gets wrong. And it breaks tap on status bar to scroll to top, to boot.
I don’t understand these scrolling libraries. The worst native browser scrolling crushes (crushes!) even the best fake scrolling JavaScript. And typical js scrolling is far from the best.
Had a similar experience on Android, but maybe it’s only supposed to make a difference in desktops?
In general, I agree with you regarding the intence page.
However, the natural scrolling has good applications in places where automatic scrolling has to happen – especially when 2 dimensions are involved. A project I worked on this summer needed to highlight the relationship between a list and an SVG, and we used automatic (smooth) scrolling to make sure the thing that was to be highlighted was actually visible
Yeah, kinda makes you wonder: if JS is in fact going to take over the world, wouldn’t it have done so by now? JS is butting up against technical limits, and I don’t see browser vendors having any real incentive to push things further. Three of the four major browser vendors (Google/MS/Apple, but not Mozilla) have a major source of income established on top of native apps. That is probably a safer revenue stream than the web can ever be.
(Though from the app economy’s POV, wouldn’t it be nice to reclaim some of the economic surplus being extracted by the app stores? I don’t think it’s possible though, I think app store success is somewhat derived from there being a consistent and optimized way to pay – i.e. a payment system monopoly is a precondition to the current native app distribution success.)
Maybe it would work to build JS a new UI toolkit that is not dependent on HTML (but polyfilled to it). Surely it would be easy to write useful apps in JS if you were targeting a different toolkit, i.e. one not hamstrung by HTML. Android showed that it is possible to write OK user facing apps in Java. It seems within reason to do so in JS as well.
Then again, I think it’s fair to say that any web developer turned app developer that cares about UX enough to switch to a more native language has already done so. And therefore the JS developers that remain have been selected down to those that don’t care about UX as much or aren’t as capable. I guess it’s a question of how many such JS developers there really are.
This looks pretty useful! Did you consider using the JSON Pointer notation for the key output? It’s not without its issues but it is “standard”.
Huh, I’ve never seen this RFC before.
You could easily change the separator to “/” in the current implementation. However, there is not (currently) an option to remove the wrapper around the keys. I think that the wrapper makes it all more easily grep-able. The RFC also does not address array indices, which makes things a little challenging.
However, adding the ability to remove the wrappers would be a simple enough change to support this kind of format if one chose.
This is the kind of distributed database I’d want to use. Dealing with data loss from extremely fringe eventual consistency bugs is not my definition of fun.
Also, the keys having a total order is really nice! :D
I don’t see FoundationDB hitting the same use cases as an eventually consistent DB. I’m not actually sure what the use case is for FoundationDB. The database does not support multiple data centers and the company actively tries to talk people out of expanding across datacenters[1]. Things that would require the kind of write performance they are offering are going to be pretty big, I would imagine. You wouldn’t host Twitter in 1 datacenter, for example (to take a comparison the author makes).
I think it’s cool that they are achieving this kind of performance. I just don’t understand want problems it’s solving. I would rather have eventual consistency where I can setup clusters around the world and sync them, giving me horizontal scalability of operations, than be stuck in 1 database cluster in one datacenter and have to get as much performance out of that as possible.
I’m not actually sure what the use case is for FoundationDB.
It goes fast, and has multi entity transactions. Already with transactions it’s better than Amazon DynamoDB, and there are lots of use cases for that. DynamoDB also doesn’t have multi datacenter replication.
Not every application needs to be multi datacenter. Tons of applications have exactly zero need for that. Not a lot of mega-scale web 42.0 apps, but those certainly aren’t the only applications that use a lot of data. Your use case is always on ecommerce (if I recall correctly), and you didn’t see a use case for Redis Cluster either, which is hardly surprising given your domain.
I would rather have eventual consistency where I can setup clusters around the world and sync them, giving me horizontal scalability of operations, than be stuck in 1 database cluster in one datacenter and have to get as much performance out of that as possible.
If these benchmarks are accurate, you don’t need multi datacenter capability for horizontal scalability. Use case wise, data exploration and analysis would be extra flashy with a database with this kind of latency and throughput. Real time analysis would be way more real time.
All that besides, the transaction coordination system is still quite interesting.
Not every application needs to be multi datacenter. Tons of applications have exactly zero need for that.
I agree. But how many of those apps need to do 14 million writes per second? It’s unclear to me who needs this kind of performance in one datacenter that doesn’t also need to scale horizontally, at the very least for availability.
If these benchmarks are accurate
I do question the results of the benchmark too. The test was only run for 5 minutes. That is far too little time for a database test. On top of that, this was in very ideal circumstances, the data fitting easily in RAM. I’d like to see a run for around a week, with data larger than can fit in RAM. Not that I think the results are a lie, more than I’m not sure it is covering a realistic workload.
More distant clients would probably be good too. They’re comparing to twitter, for example, but twitter clients aren’t all running in the same cluster. Running the DB in one AWS region and the load generator in a separate would be a good test (like say US East vs US West).
I don’t think that many need 14 million writes per second, I think it’s more than you can have less hardware and still get really good performance, with the option to scale up fairly easily in the future. You certainly don’t need to employ hundreds of servers in a base case deployment of the database, the benchmarks are more about saying that you could if you wanted to for whatever reason.
Also, RAM is the new disk. ;)
I agree with you in general, but it’s worth considering that only being in one datacenter is a resiliency hazard. If your datacenter gets smashed by a hurricane for example, you’re going to see nontrivial downtime, even if you can handle 14M writes per second.
With that said, I don’t think that foundation DB is anti-multi-DC. I think they already have some partial support of it. From their docs:
Multiple-datacenter support in this release is immature. If you are considering using multiple datacenters in production, we recommend that you contact us so we can work together for best results.
With that said, I don’t think that foundation DB is anti-multi-DC. I think they already have some partial support of it. From their docs:
I’ll believe it when I see it. The goal of the database seems at conflict with supporting multiple datacenters. And the article from FoundationDB on High Scalability seems to be trying hard to talk people out of multiple datacenters. I’ll be interested to see what they do though, clearly very capable people.
DynamoDB also doesn’t have multi datacenter replication.
DynamoDB only has multi-datacenter (well “facility”) replication. You can’t disable it. It’s not cross-region though. From documentation:
The service replicates data across three facilities in an AWS Region to provide fault tolerance in the event of a server failure or Availability Zone outage.
I’m aware, I used to work on DynamoDB before I left Amazon, I was just simplifying. The big thing people want is actually multi region replication. Having 3 data centers right next to each other makes them susceptible to the same problems. Region wide failures aren’t unheard of on AWS.
With things like the ConcurrentAdder I wonder why its a new API instead of just a new, faster implementation of AtomicInteger. The author supposes that developers should always use the adder, so why the new API?
In addition to the size argument made in a sibling comment, AtomicInteger/Long has a very different API to LongAdder. It is used for counter management because it’s the best construct available for it, but that doesn’t mean it’s a great one.
In particular you always sample when you increment an atomic type (using incrementAndGet or getAndIncrement). This means it must update inline. The adder type, which has a void increment, doesn’t have this constraint, so can defer the synchronisation cost. The actual means of implementation changes the structure’s internal representation enough that it warrants another type.
One potential problem (if it matters to you) is size: LongAdder is a lot bigger. It uses a list of packed AtomicLongs. So if you have 24 cores and high contention, your LongAdder could grow to 24 packed counters. And each counter is packed with 64 bytes to prevent false-sharing along cache lines, which makes them rather bloaty for what is basically a simple counter.
Of course, if you have 24 cores and high enough contention to cause the LongAdder to use 24 counters…you need the concurrency anyway and probably don’t care about the bloat.
If your counter is rare-contended, AtomicLong will be faster since there is a certain amount of overhead to the whole LongAdder setup.
The damn cookie notice spam won’t go away, and it’s over the contents.
Haven’t tried this one, but I usually delete the Instagram etc. overlays in developer mode.
Try clicking then dismissing the settings thing (cog icon), I had the same issue.
I found it helpful to open it in Incognito, then export it as a PDF.