A comment about CGI in general: it’s absolutely beautiful.
When I wrote my own site backend a few years back I had no knowledge about the world of interfacing webservers with code. I discovered that there were many, many methods and protocols that each webserver only seemed to support a subset of. And many people telling me that CGI was old and bad and that I shouldn’t use it.
I had a nightmare getting non-CGI things to work. I had no experience and background here, so not much of it made sense. I had presumed ‘FCGI’ was a “fixed” version of CGI, but didn’t succeed at getting it to work after following a few guides and trying a few different webservers. I gave up.
I decided I should do the opposite of modern advice. I tried CGI. And I was immediately hooked by its simplicity.
No dependencies or libraries
Works with any programming language that can print and read text
Supported by “most” webservers out there (cough nginx)
#!/bin/sh
printf 'content-type: text/html\n\n'
printf '<strong>Greetings Traveller</strong>'
printf "<p>The date and time are $(date)</p>"
The webserver itself handles all the difficult bits of the HTTP headers. You just need to provide a content-type and then the page itself. Done.
If you want to provide more (cookies, etc) you can; it’s just one more ‘printf’ line and you’re done. No libraries, no functions, no complexity. You don’t even have to parse strange constructs. Just print.
If you want to look at URL strings (eg for GET) you just need to be able to access the environment variable QUERY_STRING. If you want to access body data (eg for POST) you just need to read input. Just as if someone was sitting there typing into stdin of your program.
It does get ugly for complex or multipart POSTs. That’s where a library or program can help. But you only need to attack that once you get there.
Compare this to every other method of talking to a webserver out there:
No dependencies or libraries.
Works with any programming language ever that can read text and print text.
Zero external config other than telling your webserver to enable CGI on your file
A related story of teaching
A few months back I was helping some students with their website project. They were new to web development and had been recommended to use flask, a python library that acts as a webserver and webserver interface all in one. They were having extreme difficultly wrapping their heads around many concepts. Notably:
Templates
Serving of static files (like stylesheets)
Cookie handling
Mapping of URLs to files, functions and directories.
Many of their problems stemmed from them not knowing how HTTP worked in the first place, so I was teaching them this. What made this process horrible was then also trying to find out how and then explain how flask abstracts these concepts into its own processes and functions. I could understand how to beginners like them it seemed completely opaque.
They thought pages were unreadable objects generated by the templating code, and that the templates themselves were sent to the user’s browser along with the page. They thought cookies were handled and stored by the webserver as well as the client. The way flask’s functions worked and the examples they followed suggested this to them.
If I’m ever in the situation again of helping new people learn web technology then I’m going to get or convert them to use CGI right off the bat. It’s easier to teach, it’s easier to understand, easier to get working on most webservers and isn’t locked in to any particular language or framework.
The only downside of CGI that I know about is the fact it starts a new process to handle each user request. Yes that’s a problem in big sites handling hundreds or thousands of visitors per second. But by the time a new student gets to running a big site they will have already encountered many, many other scalability issues in their code and backend/storage. Let alone teaching them database and security concepts. There’s a reason we have quotes like “premature optimisation is the root of all evil”.
I don’t think students new to webdev should be started on anything other than CGI. They can use any language they want. They can actually understand what they’re doing. And they’re not hitting any artificial barriers or limits set by frameworks or libraries.
Final notes
The whole idea that “CGI should be dead” makes little sense from my context and point of view. I run my own site, help maintain a few others and try to assist others in learning and coping with webdev.
I think the “CGI should be dead” makes sense only in the context of very high workload sites. Whilst these handle a large percentage of the web’s total traffic, the percentage of people actually running these sites is small. Different units: traffic of visitors vs people running sites. I think we confuse them.
It’s too easy to get caught up in “professional syndrome”, where you look up to the big players and trust in their opinions. But you also need to understand that their opinions are based on their current experiences, which are often a world away from what the rest of us should be worrying about.
If a captain of a battleship says that cannons are his biggest problem then you shouldn’t try to learn about and use cannons to build your first ship. You should then realise only a tiny fraction of ships need them, even the really big ones.
FWIW this may be the single most thorough and thoughtful comment response I’ve seen on Lobsters. Thank you for that.
I agree that CGI’s simplicity is beautiful, and as much as you’re suggesting that libraries aren’t required (and they aren’t) if you look at something like CGI.pm it’s an incredibly thin wrapper around CGI. A little bit of convenience, some simple templating for generating HTML. This was a winning recipe for thousands people designing interactive websites for literally decades!
I’ll have a look at CGI.pm. One thing in particular that I found hard was decoding multipart form data; so I ended up writing a standalone C program that does nothing but split these up into separate files.
CGI is pretty awesome, and I’ve always wanted to find a nice way of using Python with it
Unfortunately there’s a bit of an awkwardness when trying to pair CGI with something like Django. Having to set up the environment over and over. Also there’s the basic issues of URL routing being coupled with files…
I’m sure there’s some nice style of dealing with this, but I haven’t found it yet.
Yeah I actually rather wonder if many of the advantages that caused folks to go the in-process like mod_perl, PHP etc have been rendered pointless by modern processor and IO.
I think a lot of those weren’t even CGI per se, but just that perl and php (and other interpreted languages) have nasty startup costs. I often do traditional CGI programs in the natively-compiled D language and while it doesn’t win any benchmarks, it is also good enough for a LOT of things - and very reliable, thanks to process isolation.
I’ve also been playing around with Go’s net/http/cgi package, which is quite nice, since it uses the same backend as the regular http, as well as the fcgi package. One can even bundle them together into one binary, that chooses what to done depending on the file name (example for a mini lobste.rs clone I wrote a while ago).
And from my experience, which wasn’t quite the worst, it was quite enough. I sometimes have the feeling that modern web frameworks have created a fear of these more simple and often sufficient solutions (for smaller to mid-size usecases), to promote their own projects - unrightfully.
Yeah, the library I wrote for D also uses the same interface for cgi, fcgi, scgi, and http (both pre-fork processes to provide some segfault isolation as well as simple multiplexing, and one-thread-per-connection primarily for compatibility on non-Linux systems where fork doesn’t work the same way) implementations, so swapping out is often as simple as a recompile.
I don’t even think CGI is really dead, but rather just slightly extended or wrapped by everyone individually. Python’s WSGI and Ruby’s rack show their cgi heritage and compatibility, etc.
It’s a remarkably well designed tool for HTML templating. The only time I’ve ever successfully built a ‘dynamic’ website was using CGI.pm. Nowadays the complexity makes my tiny brain ’splode :)
FWIW I’ve heard good things about the Mason web framework, but my perl-fu is WAY rusty.
I was thinking about this the other day as I FINALLY ditched my ancient Lincoln Stein CGI.pm book.
So today I googled it and this came up. Kind of impressive that it survived in the perl distro until just 3 years ago! That’s incredible longevity for that kind of software IMO.
Huh? But why would they remove it ? OK, it’s old. That doesn’t mean it doesn’t work, and it has precisely the problem that he’s pointing out – scripts will break on old web hosts. I assume those hosts will notice and simply backport CGI.pm.
There are good technical reasons for this. CGI is a dying technology. In 2015, there are far better ways to write web applications in Perl. We don’t want to be seen to encourage the use of a technology which no-one should be using.
A comment about CGI in general: it’s absolutely beautiful.
When I wrote my own site backend a few years back I had no knowledge about the world of interfacing webservers with code. I discovered that there were many, many methods and protocols that each webserver only seemed to support a subset of. And many people telling me that CGI was old and bad and that I shouldn’t use it.
I had a nightmare getting non-CGI things to work. I had no experience and background here, so not much of it made sense. I had presumed ‘FCGI’ was a “fixed” version of CGI, but didn’t succeed at getting it to work after following a few guides and trying a few different webservers. I gave up.
I decided I should do the opposite of modern advice. I tried CGI. And I was immediately hooked by its simplicity.
For those not in the know, a fully working CGI script is as simple as this:
The webserver itself handles all the difficult bits of the HTTP headers. You just need to provide a content-type and then the page itself. Done.
If you want to provide more (cookies, etc) you can; it’s just one more ‘printf’ line and you’re done. No libraries, no functions, no complexity. You don’t even have to parse strange constructs. Just print.
If you want to look at URL strings (eg for GET) you just need to be able to access the environment variable QUERY_STRING. If you want to access body data (eg for POST) you just need to read input. Just as if someone was sitting there typing into stdin of your program.
It does get ugly for complex or multipart POSTs. That’s where a library or program can help. But you only need to attack that once you get there.
Compare this to every other method of talking to a webserver out there:
A related story of teaching
A few months back I was helping some students with their website project. They were new to web development and had been recommended to use flask, a python library that acts as a webserver and webserver interface all in one. They were having extreme difficultly wrapping their heads around many concepts. Notably:
Many of their problems stemmed from them not knowing how HTTP worked in the first place, so I was teaching them this. What made this process horrible was then also trying to find out how and then explain how flask abstracts these concepts into its own processes and functions. I could understand how to beginners like them it seemed completely opaque.
They thought pages were unreadable objects generated by the templating code, and that the templates themselves were sent to the user’s browser along with the page. They thought cookies were handled and stored by the webserver as well as the client. The way flask’s functions worked and the examples they followed suggested this to them.
If I’m ever in the situation again of helping new people learn web technology then I’m going to get or convert them to use CGI right off the bat. It’s easier to teach, it’s easier to understand, easier to get working on most webservers and isn’t locked in to any particular language or framework.
The only downside of CGI that I know about is the fact it starts a new process to handle each user request. Yes that’s a problem in big sites handling hundreds or thousands of visitors per second. But by the time a new student gets to running a big site they will have already encountered many, many other scalability issues in their code and backend/storage. Let alone teaching them database and security concepts. There’s a reason we have quotes like “premature optimisation is the root of all evil”.
I don’t think students new to webdev should be started on anything other than CGI. They can use any language they want. They can actually understand what they’re doing. And they’re not hitting any artificial barriers or limits set by frameworks or libraries.
Final notes
The whole idea that “CGI should be dead” makes little sense from my context and point of view. I run my own site, help maintain a few others and try to assist others in learning and coping with webdev.
I think the “CGI should be dead” makes sense only in the context of very high workload sites. Whilst these handle a large percentage of the web’s total traffic, the percentage of people actually running these sites is small. Different units: traffic of visitors vs people running sites. I think we confuse them.
It’s too easy to get caught up in “professional syndrome”, where you look up to the big players and trust in their opinions. But you also need to understand that their opinions are based on their current experiences, which are often a world away from what the rest of us should be worrying about.
If a captain of a battleship says that cannons are his biggest problem then you shouldn’t try to learn about and use cannons to build your first ship. You should then realise only a tiny fraction of ships need them, even the really big ones.
FWIW this may be the single most thorough and thoughtful comment response I’ve seen on Lobsters. Thank you for that.
I agree that CGI’s simplicity is beautiful, and as much as you’re suggesting that libraries aren’t required (and they aren’t) if you look at something like CGI.pm it’s an incredibly thin wrapper around CGI. A little bit of convenience, some simple templating for generating HTML. This was a winning recipe for thousands people designing interactive websites for literally decades!
Thanks feoh.
I’ll have a look at CGI.pm. One thing in particular that I found hard was decoding multipart form data; so I ended up writing a standalone C program that does nothing but split these up into separate files.
CGI is beautiful and perfect and it is sad how neglected it is nowadays.
CGI is pretty awesome, and I’ve always wanted to find a nice way of using Python with it
Unfortunately there’s a bit of an awkwardness when trying to pair CGI with something like Django. Having to set up the environment over and over. Also there’s the basic issues of URL routing being coupled with files…
I’m sure there’s some nice style of dealing with this, but I haven’t found it yet.
Yeah I actually rather wonder if many of the advantages that caused folks to go the in-process like mod_perl, PHP etc have been rendered pointless by modern processor and IO.
I think a lot of those weren’t even CGI per se, but just that perl and php (and other interpreted languages) have nasty startup costs. I often do traditional CGI programs in the natively-compiled D language and while it doesn’t win any benchmarks, it is also good enough for a LOT of things - and very reliable, thanks to process isolation.
Why would you start a CGI script on request when you can just launch a whole container now? /s :P
I’ve also been playing around with Go’s net/http/cgi package, which is quite nice, since it uses the same backend as the regular http, as well as the fcgi package. One can even bundle them together into one binary, that chooses what to done depending on the file name (example for a mini lobste.rs clone I wrote a while ago).
And from my experience, which wasn’t quite the worst, it was quite enough. I sometimes have the feeling that modern web frameworks have created a fear of these more simple and often sufficient solutions (for smaller to mid-size usecases), to promote their own projects - unrightfully.
Yeah, the library I wrote for D also uses the same interface for cgi, fcgi, scgi, and http (both pre-fork processes to provide some segfault isolation as well as simple multiplexing, and one-thread-per-connection primarily for compatibility on non-Linux systems where fork doesn’t work the same way) implementations, so swapping out is often as simple as a recompile.
I don’t even think CGI is really dead, but rather just slightly extended or wrapped by everyone individually. Python’s WSGI and Ruby’s rack show their cgi heritage and compatibility, etc.
It probably is dying, but I feel I ought to point out that OpenBSD’s man.cgi uses CGI and OpenBSD has added slowcgi(8) to base.
I use
CGI.pmjust to generate HTML in an efficient way from Perl. Guess it’s time for me to find another way to do that now!It’s a remarkably well designed tool for HTML templating. The only time I’ve ever successfully built a ‘dynamic’ website was using CGI.pm. Nowadays the complexity makes my tiny brain ’splode :)
FWIW I’ve heard good things about the Mason web framework, but my perl-fu is WAY rusty.
Mason does sound like the new hotness.
I basically just wanted to do stuff like transform a Perl list into HTML entities (
ol,ul,table) and CGI.pm had tools for that…I was thinking about this the other day as I FINALLY ditched my ancient Lincoln Stein CGI.pm book.
So today I googled it and this came up. Kind of impressive that it survived in the perl distro until just 3 years ago! That’s incredible longevity for that kind of software IMO.
Huh? But why would they remove it ? OK, it’s old. That doesn’t mean it doesn’t work, and it has precisely the problem that he’s pointing out – scripts will break on old web hosts. I assume those hosts will notice and simply backport CGI.pm.
This is not a good reason.
EDIT: I found this in the comments: https://news.ycombinator.com/item?id=10757628
I think that is pretty dumb, but I’m not interested in rehashing a 3-year-old argument for a language I don’t use :-)
Good point! Are there any security ramifications that make CGI less desirable than any of the mod_blah methods?
The copyright on the source is 1995. So, it took 20 years for this module to be removed. That’s a damn impressive run in technology terms :)