Threads for cliffordfajardo

  1. 3

    I don’t have too much to add (will read article later..), but I want to cross reference a thread on this topic which got a lot of responses from lobster community “how much logic should I keep at the database vs application layer?”

    1. 1

      Adding more to this thread: below is the redacted conversation that inspired this question. My friend & I both work{ed} at FAANG companies, don’t have undergraduate degrees, we took the bootcamp path a few years ago

      Conversation: 👤Friend:

      I’m thinking about doing Columbia University’s General Studies undergrad program, which is tailored for working professional’s who come from non-traditional backgrounds (Ex: starting college later in life, have lots of life experience, looking to get back into academics).

      👤Me:

      I attempted doing UC Berkeley undergrad part-time, while working full time and I’ll share my experience. For context I got a full time programming job at Salesforce right after graduation from a coding bootcamp in 2016 in the summer right before the fall I was admitted as an incoming 3rd year undergraduate transfer (my major was Sociology..nothing to do with CS). Berkeley doesn’t an official program for part-time undergrad students, that I’m aware of, but they do for master’s + level degrees.

      My personal experience trying to get undergrad degree + work full time:

      • I gave UC Berkeley part-time an attempt 1 semester by taking only 2 classes: CS61A & an elective. It wasn’t hard, but it was just MASSIVELY time consuming what they required.
      • Trying to make a curriculum designed for undergrad students , who 99% of them are full-time students, who typically live on/close to campus work, is a lot different than a working professional who is in the field already and has working full-time job; there are things like: in personal labs 2x a week on different days than the actual class, group projects etc… Things I didn’t like about my personal experience:
        • I felt like I was grinding for grades? Why? Well if you’re investing in an undergrad experience, you want to do decently well to perhaps leave the opportunity for grad school open.
        • doing these strange problem sets & diagrams to prove I knew how to reverse a linked list or to understand the time complexity of a binary tree felt like a big time sink; definitely valuable for my peers who were not already in the field or had exposure to these concepts

      I’m not saying traditional education isn’t good, but I think for a bootcamp graduate from a caliber coding bootcamp like “Hack Reactor”, “Codesmith”, “Fullstack Academy of Code” etc… its not too useful to go back to a traditional undergraduate setting when there are now respectable good alternatives educational institutions like Bradfield nowadays.

      • For those who don’t know there are coding bootcamps that take you from 0-80, start from nothing, to other program’s that take you from 20-100. In the latter bootcamps, you need to come in with a baseline level, there is pre-courswork, the curriculums cover applied data structures/algos, practical recursion, system design, group projects etc. Me & my friend went to one of the latter bootcamps

      There are few people championing the non-traditional path in research (as mentioned in the original question post) that I know of, but I wish I knew more folks:

      Jason Benn - a classmate of mine at Bradfield (bootcamp grad too) went from Devbootcamp (2013) to Machine Learning engineer and researcher. Fun fact: he’s the programmer that Cal Newports mentions in his book “Deep Work”.

      Richard Artoul (HRX & Bradfield alum) — I’m not sure if Richard is writing research papers, but this doing pretty deep stuff in Database engine internals. This is some of his work below:

      Kyle Kingsbury – does distributed systems research (creator of jepsen - A framework for distributed systems verification, with fault injection) I think he holds an undergrad degree in physics, and no postgrad degree (doesn’t have LinkedIn)

      Personal Opinion(s):

      • If you’re in the situation of being a software engineer in the field already without a “formal education” and you want to dive deeper into the topics that the top computer science departments in the world have done an excellent job at identifying over the past few decades, consider Bradfield (not endorsed by them, I like their mission + took 6 classes here)

        • I took 6 classes at Bradfield covering all the major topics of computer science you would typically cover at places like UC Berkeley, Stanford you’re last 2 years of undergrad. I enjoyed Bradfield a lot due to my peers, who were all working professional software engineers (ranging from SWE - Sr.Staff levels, Managers etc) who were curious/passionate about learning; i observed that their “skin in the game” factor maybe was higher than undergraduate students – you don’t accidentally sign up to take on additional studies accidentally on top of having a full-time job or family) the teachers and environment were great as well.
      • forking 10-50k on an undergrad degree, even if it is a brand name, not only sounds expensive financially but also from a time investment perspective, especially if you have a job & want to continue doing well, and/or have other obligations like a family.

      • If you want a fancy master’s degree, I know Oxford has an undergrad program that doesn’t require an undergrad degree; you just need a strong amount of work experience, demonstrated work work & recommendations. A friend of a friend I know worked at SOME_FAANG_COMPANY , did this It costs like 50k though and its a huge time investment outside of work, and again, I’m not a fan of the curriculum, since its very general IMO. If you’re chasing prestige, I think its not a good use of your time to go back to undergrad

      1. 1

        My view seems a bit different than what’s already here so: choose one or the other. There’s nothing wrong about putting all the logic in the app. There’s nothing wrong about having the app interface with the database through views and stored procedures that do all the business logic. The pain arrives with trying to find some threshold / splitting the responsibility in an arbitrary way that’s more justified to one person than another. Choose one, be consistent and you’ll be fine.

        1. 1

          I think you can’t go terribly wrong with either choice as you mentioned; some other responses in here did mention interesting use cases for putting it in the DB layer (but again context informs so much) like

          Choose one, be consistent and you’ll be fine. this is a good reminder ^

        1. 2

          Q1: hard question. Basically use heuristics to find the best fracture planes depending on team organization/ownership + understanding of the solution + maintenance burden + coupling + technical constraints.

          Depending on what you decide to implement in the database, it will be able (or not) to scale computations over data. Having remote stateless applications allows to scale computations at the expense of bandwidth. my 2 cents in 2 minutes.

          Q2: https://rob.conery.io/2015/02/21/its-time-to-get-over-that-stored-procedure-aversion-you-have/

          1. 1

            Basically use heuristics to find the best fracture planes depending on team organization/ownership + understanding of the solution + maintenance burden + coupling + technical constraints.

            Well articulated & succinct outlining of lots of the variables involved. Thanks for the stored procedure article

          1. 2

            Sometimes it is useful or even required from the performance point of view to have computation near the data just to not transfer it through the network - not talking about data validation but, for example, reporting. There are other solutions to reporting problems but most straightforward and requiring often less code is to just run a stored procedure. With all of the drawbacks of stored procedures.

            1. 2

              Sometimes it is useful or even required from the performance point of view to have computation near the data just to not transfer it through the network

              This is a really good point on data locality. If your database server’s hardware specs are significantly better than your app servers, then this could help improve performance a lot

            1. 7

              I would argue that the DB is responsible for data integrity and thus should validate the format of an email address if possible.

              IMHO the DB is responsible for data integrity, data formatting, data storage, and data extraction and transformation (if possible).

              That last point is something that I’ve seen ignored a lot. I’ve seen countless apps pull a huge amount of data from the DB, do a trivial transformation, and store it all back when the DB could have done it faster, safer, and more clearly in situ.

              1. 1

                I would argue that the DB is responsible for data integrity and thus should validate the format of an email address if possible.

                I like this perspective, adding on to the “if possible” statement: If its not easy to enforce in DB, its hard to modify/contribute/maintain, it’s might be a good idea to handle it in the code if your language of choice can do a better job Interestingly after reading so many of the responses in the thread, I can see the benefit of putting the logic at the DB layer and/or app code

              1. 8

                Why can’t a db validate the format of an email address? The pattern is known, even if the RFC is crazy compared to real email addresses.

                1. 10

                  I recommend using the regex from HTML5 spec. Much simpler and very relevant as that’s what browsers use for input=email

                  edit: link: https://html.spec.whatwg.org/multipage/input.html#valid-e-mail-address

                  1. 9

                    For the most part, the validity of the email address itself is not an issue, it’s whether the user can be contacted by it. Knowing this requires a dance of sending the email and verifying that the user got it, which is way beyond the purview of the database. I usually don’t see a point in going beyond NOT NULL on a database field for email.

                    1. 2

                      I think it makes sense to (perhaps in the UI) just do a sanity check, like an @ sign, just to fix up any big fat-fingers. Before the TLD explosion, it was pretty easy to ensure the TLD also, but that’s over(yes purposely ignoring UUCP/bang-path type addresses).

                      Otherwise I basically agree with you. NOT NULL UNIQUE forces the email field to do what you need it to do that you can control: be a unique identifier. If it happens to also work for communication, bonus! :)

                      1. 1

                        add an “validated” field and it’s good to go..

                      2. 1

                        Why can’t a db validate the format of an email address?

                        You gotta good point; you can validate an email address at the DB layer for sure. I looked into this a bit more now that you mentioned an RFC

                        There is a really good response here by this DBA suggesting similar thought of using the HTML5 email regex spec “if its good for HTML5/browser its good for the rest of us” - https://dba.stackexchange.com/a/165923

                      1. 2

                        I find this interesting as over the past few years, there has been a rise in block based editors / apps. For example:

                        • Notion implements these concepts (not the protocol mentioned here) to build their block based editor built on top of markdown primitives. You can drag videos, images, pdf files, tables and so many other things. I love this model actually. See “Data Model behind Notion Engineering blog post
                        • Microsoft is building their own Notion, Coda, Confluence alternative built on block primitives called Microsoft Loop. I’m very excited about this. Currently we use confluence/jira at LinkedIn (where I work) and this could replace that. It integrates with all of Microsoft’s other products like: outlook, Teams, and more.

                        As someone who is a note maker, I’m most fascinated about building block based experience that compile down to simple primitives like pieces of markdown. I take notes in my text editor in markdown files. Sometimes I wish I could toggle into an editor mode like notion & quickly revert back to text-editing mode.

                        I found this OS VueJS library which recreates the notion block editor experience, one day..to hack on it https://github.com/CedarXi/All-in-one