1. 102
  1. 17

    However, it also defines operator* and operator->. These methods also provide access to the underlying T, however they do not check if the optional actually contains a value or not.

    Wat?

    1. 15

      Using operator* or operator-> on an optional that doesn’t contain a value is undefined behavior, see: spec, cppreference

      I’ve been about to migrate a bunch of APIs from std::unique_ptr<T> to std::optional<T> and now I’m really not feeling as great about that plan.

      1. 19

        On the face of it this seems like an insanely bad API design decision by the committee.

        1. 5

          What should it do? Throwing exceptions would make the API useless to many large consumers. Default-constructing the value type would probably be worse. The C++ type system just isn’t very good.

          1. 25

            What should it do?

            not offering those operators at all and instead forcing the usage of map, flatmap or get_or_default like most languages do.

            We opensourced a library that implements a more functional API here

            1. 2

              What should it do?

              not offering those operators at all and instead forcing the usage of map, flatmap or get_or_default like most languages do.

              That’s unacceptable when you know the optional has a value and performance is critical. Nothing is stopping you from implementing “get_or_default” on top of the operations provided, but you cannot implement operator* as it currently is on top of “get_or_default”

              1. 4

                That’s unacceptable when you know the optional has a value and performance is critical.

                C++ is a zero cost abstraction language and lambdas get inlined. Unless the optional is implemented via a vtable, you should not see any overhead.

                1. you might be right today but what about tomorrow? - will your assumption still be true if someone else will modify some (distant) part of the codebase?

                2. If you know that an optional will always contain a value, why is it in an optional in the first place?

                Nothing is stopping you from implementing “get_or_default” on top of the operations provided, but you cannot implement operator* as it currently is on top of “get_or_default”

                Well the argument is operator* you should not exist as it eventually leads to unsafe code and bugs. You may always know when you can get away of using operator* without prior checks and may not need such construct but most people get it wrong at least once.

                If you work in a team you will realize those unsafe usages become a liability as they are prone to break due to distant changes. As a result this is not very social: this practice pushes the responsibility of debugging or verifying your assumptions to everyone who is contributing to the codebase. Every Time someone works on a feature and coincidentally breaks the unsafe code will also need to fix it up; it becomes his responsibility.

                Lastly, just for the sake of argument: In case your optional implementation does not include a footgun, this is how you could build one:

                T const& optional<T>::operator*()const {
                   return this->get_or_default(*(T*)nullptr);  // that's fine it's just undefined behaviour 
                }
                
                1. 2

                  You shouldn’t be forced to check if the option is filled each time you access it. One check is enough, then you can assert that the option is filled in all following code paths. If you don’t want to do that, force your team to always use “get_or_default” but don’t force the world to work like your team, not everyone has your performance constraints.

                  1. 1

                    You shouldn’t be forced to check if the option is filled each time you access it.

                    You only test once. If you are worried that running map or get_or_default test for the content multiple times, don’t worry, the compiler will optimize it away: https://godbolt.org/z/7RGUCf

                    1. 1

                      That’s not true in all cases.

                2. 2

                  If you know a value exists and performance is critical …why are you using std::optional in the first place?

                  By this logic, you might as well demand that std::map should have implemented operator[] not to take const Key& key but instead an already hashed bucket identifier, and if there’s nothing in that bucket, or not what you were expecting, oh well, have some undefined behaviour?

                  It’s silly. If you don’t want the overhead of a particular container, don’t use that container.

                  Insisting that std::optional container has to have a hole in it … either you want an Option or you want a raw pointer. “I want an Option but it has to behave exactly like a bare pointer” is an incoherent position for an API trying to satisfy everyone and ending with something useful to no one. “It’s safe as long as you always 100% without fail manually check it before dereferencing, otherwise undefined behaviour” adds nothing that wasn’t already available.

                  1. 2

                    If you know a value exists and performance is critical …why are you using std::optional in the first place?

                    What if you already checked it once? Why should you be forced to check each time you access the contained value?

                    Insisting that std::optional container has to have a hole in it … either you want an Option or you want a raw pointer.

                    It’s not a raw pointer, it’s a boolean paired with a value. There’s no necessary indirection.

                    1. 2

                      What if you already checked it once? Why should you be forced to check each time you access the contained value.

                      Then extract the value the first time? Why would you design an entire API around “people might unnecessarily check for whether some value is present multiple times”? Leave Nones at the edges of your program. Once you’ve determined that there is Some(x), everything else should be dealing with x, not the Option it started in.

                      The question, if anything, illustrates exactly why these are bad operators to have in the API.

                      Without them, Options are cumbersome enough that you very quickly realize slinging them around in the core of your program is silly; you extract the present value, substitute a default, or deal with an unsubstitutable None at the edge, and design everything else in terms of the value.

                      With these operators apparently a nonzero subset of people think “I should just leave the value in the Option and touch its raw memory because “I know” I checked for presence once upstack”. Which works until someone refactors something and suddenly there’s a codepath that pulls the trigger on your footgun.

                      Good API design guides the user towards good usage. This clearly doesn’t.

                      It’s not a raw pointer, it’s a boolean paired with a value. There’s no necessary indirection.

                      It’s a raw pointer (T*) you’re getting back in the case of operator* and operator->, the operators this entire thread of posts is discussing. If you’re going to blindly call those you could equivalently have just slung around raw pointers wherever you acquired the option to begin with.

                      1. [Comment removed by author]

                        1. 1

                          Hmm I’m starting to doubt that you fully understand how optional works.

                          Spare me your condescension, oh self-appointed C++ guru. I’ve gotten paid more than enough for it over the years.

                          I said “extract the value” as in, store a ref or move it or if it’s cheap copy it, but don’t just keep passing the optional around calling value or (god help you “because you cross your heart checked it upstack”) operator->. Luckily value safely gives you the same T& that operator* unsafely gives you. You can even move it safely, avoiding the need for expensive copies, if necessary.

                          A value is stored in the optional, not a pointer. Extracting the value results in copying a potentially large block of memory. There are many legitimate instances where it makes more sense to access the value from the optional without copying it.

                          Again, if you’re in a situation where: you have an optional value AND it’s too expensive to copy AND you want a reference to the memory in the optional AND you don’t want to pay for the single solitary check one time check when you call value to extract the value THEN either you’re terminally confused because you somehow think:

                          if(opt.has_value) { 
                             SomeVal& v = *opt
                             // do something with v 
                          } else { some other logic }
                          

                          is any amount of checks or derefs or copies faster than the safe alternative of

                          SomeVal& v = opt.value_or(/* some other val */)
                          // do things with v
                          

                          OR you absolutely can’t even deal with the overhead of the single check you do have so you’re intending to blindly do

                          opt->foo
                          opt->bar
                          

                          in which case, again just start with a pointer to the possibly undefined memory in opt and save the middleman.

                          1. 0

                            Hmm I’m starting to doubt that you fully understand how optional works

                            Spare me your condescension, oh self-appointed C++ guru. I’ve gotten paid more than enough for it over the years.

                            It’s not condescension, you were arguing along the assumption that optional as it currently exists has no benefit over using a raw pointer, completely ignoring the fact that it gives optional semantics to value types. That isn’t a dig at your general C++ knowledge, just your understanding of optional.

                            You can even move it safely, avoiding the need for expensive copies, if necessary.

                            std::move() is intended to speed up value types that contain pointers like std::vector. It doesn’t make moving a large struct fast.

                            A value is stored in the optional, not a pointer. Extracting the value results in copying a potentially large block of memory. There are many legitimate instances where it makes more sense to access the value from the optional without copying it.

                            Again, if you’re in a situation where: you have an optional value AND it’s too expensive to copy AND you want a reference to the memory in the optional AND you don’t want to pay for the single solitary check one time check when you call value to extract the value THEN either you’re terminally confused because you somehow think:

                            There are serious caveats in defining value_or() in this construction: “SomeVal& v = opt.value_or(/* some other val */)”. How would you define it?

                            in which case, again just start with a pointer to the possibly undefined memory in opt and save the middleman.

                            I’ve already said that the point of optional is to give optional semantics for those cases when you must use a value type and not a pointer. Do you understand what that means? What if you can’t use the heap/a pointer?

                3. [Comment from banned user removed]

                  1. 7

                    Could you clarify what you mean by idiomatic? What design would be both idiomatic but not permit dereferencing a null/uninitialized pointer?

                    1. 23

                      dereferencing a null/uninitialized pointer is idiomatic C++

                      1. 3

                        and segfaults…

                        It’s true that nullpointers may be valid addresses on some architectures but we should always model the semantics correctly in our programs.

                        But what does not make sense is dereferencing an empty optional/smart-ptr. If a null-pointer is a valid and intended value than you should assign it to eg. your optional/smart-ptr.

                4. 9

                  I mean, the semantics of Option in the abstract just aren’t safely compatible with an operation that blindly assumes Some(x) is present and directly grabs x’s memory, so the choices are either:

                  1. use exceptions to avoid unsafe accesses at runtime, which as you say, not every codebase will appreciate

                  Or

                  1. just don’t offer those operators on Option. They fundamentally compromise the entire point of the type. If you want to make blind assumptions about memory being initialized, use a raw pointer; Option should enforce its safety benefits.
                  1. [Comment from banned user removed]

                    1. 13

                      It offers those operators

                      It offers implementors the choice to implement those operators on their types. There is absolutely no requirement that all types do so, let alone do so in a way that provides unsafe semantics. Their presence on std::optional makes little sense given the semantics of Option as an Abstract Data Type

                      1. 2

                        Feels like Java 8’s Optional type all over again.

                        1. 2

                          As bad as Java 8’s Optional#get is, it at least always aborts control flow via an exception if an empty value is forced unwrapped.

                          From what I can see, the undefined behaviour of C++ theoretically allows an implementation to not abort control flow and perhaps use zero values or something, which makes reasoning about control flow invariants in portable code harder.

                          1. 1

                            Why is Java’s Optional bad? Having used both Java and Rust, the only thing I really miss is pattern matching, which is a separate concern from the type itself.

                            (I already know about the complaint that the optional itself can be null, but… baby steps, I guess.)

                            1. 2

                              “Bad” is honestly too strong a word for it; it got a lot right.

                              That said, it gave a force-inwrapping operation the name get which is both too easy to discover and doesn’t signify the exceptional case well enough. A member of the JDK team admitted it was a mistake which is why it’s being deprecated in a future JDK for a more verbose name.

                              The way it handles nulls is also annoying. Not because optionals can themselves be null like any other object, but because returning null in areas like mapping lambdas implicitly converts into empty optional rather than a present optional of a null. While getting rid of nulls like this seems like a good idea initially, it breaks the monadic laws which breaks composability in places.

                              This article from Atlassian details the problem: https://blog.developer.atlassian.com/optional-broken/

                              1. 1

                                Eh. I think of the Java 8 stuff as occupying an intersection of two similar languages that don’t share all of their idiomatic patterns. I know that winds up resulting in yet more inconsistencies in spots, but I don’t know how much that bites people in practice.

                                Then again, I acknowledge most of the stuff I’ve been working with is greenfield— there’s nothing to accidentally refactor the wrong way.

                      2. 1

                        It offers those operators, you can use them if you want to

                        meaning it offers you the option to introduce bugs to your codebase if you want to?

                    2. 3

                      What should it do? Throwing exceptions would make the API useless to many large consumers.

                      And undefined behavior makes it usable?

              2. 12

                I recently introduced a subtle bug in a C++ codebase by failing to hold a union just right. The code misbehaved confusingly. It actually crashed clang’s ASan. Once I worked out WTF was going on I understood why it was doing what it was doing but until then it was mysterious, subtle and frustrating.

                1. 18

                  There is a palpable sense that the entirety of the existing C++ tooling can barely manage to keep it’s head above the water.

                  1. 2

                    It actually crashed clang’s ASan.

                    Meaning that the compiler crashed but only with -fsanitize=address? Can you share more details about this?

                    1. 1

                      No, it crashed at runtime in the asan runtime instead either telling me what was wrong. Without asan it just kept going with corrupted state and crashed later. So asan sort of helped…

                  2. 10

                    My conclusion, based on reviewing evidence from numerous large software projects using C and C++, is that we need to be migrating our industry to memory safe by default languages (such as Rust and Swift).

                    I totally support this. C(++) is awesome for being so bare bones when working with low level stuff. But I feel software projects not needing such functionality could greatly benefit from safe languages.

                    1. 15

                      Could you elaborate on what you consider low-level stuff, where Rust, Ada, or other safe languages are not an option? You can compile Rust to STM32 microcontrollers with 20KB RAM and 64KB flash (with no_std), Rust has SIMD intrinsics for various platforms, etc.

                      To me, the major reason to go for C or C++ are ecosystems in which they are traditionally strong. gcc targets a lot of platform. Many embedded toolchains are only available for C or C++. The best traditional GUI libraries are in C or C++ (e.g. Qt). And (obviously) a gazillion lines of existing code are in C or C++, from open source unix kernels to deeply proprietary systems that will be used and maintained for decades to come.

                      Even though it may not be the (only) goal of the C++ standards committee, I see Modern C++ as a band-aid for maintaining C++-based systems that will be around for a long time, than a serious contender for modern, safe languages.

                      1. 1

                        Of course, ‘low level’ is very vague. I meant projects in which you want to use raw pointers and want to calculate with them, want to use CPU registers, want to have control over the produced asm, projects for very constrained platforms or obscure architectures, and so on. I do not necessarily mean other languages aren’t an option. I’m sure languages like Rust could replace a lot of C(++) code bases, even for these ‘low level’ things. In fact, I’d totally go for replacing such projects with a Rust variant if possible. But at the same time I can understand why some might choose to go with C(++).

                        1. 9

                          As far as I can tell as soon as your code is complex enough to make you want to move from C to C++ then you should probably reach for something safer. It’s hard to keep both the complexities of the invariants you need to manually ensure in your head while solving complex software problems.

                          1. 4

                            ATS is an option here. It allows type safe use of low level pointers, including addiction, dereferencing, etc.

                        2. 2

                          C and C++ are completely different languages at completely different ends of a lot of spectrums. C is certainly bare bones, but C++ is not. What do you mean by ‘C(++)’?

                        3. 4

                          Not an expert in D, but D had also been addressing this problem head on https://dlang.org/spec/memory-safe-d.html

                          As far as C++, I wish there would be a switch –no-undefined-behaviour-features. That we could choose to use for a project (and a package system would only allow to use 3rd party packages that support the switch as well).

                          it seems that leaving behaviour undefined (and declaring it in specs without any user-choseable recourse) should not be done.

                          1. 4

                            The last time I wrote in C++ was probably 10 years ago. It was a server with libpurple that can do chat on multiple platforms. I have since then intentionally kept my self isolated and away from jobs/projects that involved C++. Not to say it is a bad language but the compile errors enter unreadable territories and syntax sometimes yields hard to trace code.

                            D lang or Go lang are some modern alternatives with Go being obviously more adopted. However my eyes are out for Rust, which in my opinion is still not the silver bullet but departure in right direction.

                            1. 2

                              I imagine experienced C++ programmers will now also look out for these new pitfalls, and just add them to their (already comprehensive) mental library of C++ pitfalls.

                              The cost of buying into C++ is high, so the will to abandon it is proportionally low.

                              1. 2

                                I see an interesting contradiction by those arguing that such failures are simply C++ being misused: they often argue that Modern C++’s biggest advantage over plain C is the myriad features that make development safer, like std::string and smart pointers.

                                Couldn’t their arguments against Rust also be levelled against using C++ over C? After all, who needs the safety afforded by destructors and std::vector if we can simply program in C carefully with popular collection and string libraries?

                                It seems that the most compelling point in favour of using Modern C++ over C is also the best argument for using Rust over Modern C++.

                                That’s not even taking into account the Modern C++ features mentioned like reference capturing lambdas and classes like std::span that add even more unsafe foot guns over good ol’ K&R C.

                                1. 2

                                  This post is obviously written from the perspective of someone who cares about safety and security. Safety is a very important ‘top’ to the system but there are others which can be more important depending on what the user values. The software can be as safe as you want, but if it doesn’t solve the problem I need the software to solve, then it’s useless to me. If safety concerns are preventing me from writing software that is useful to people, then it’s not valuable. In other words, sometimes ‘dangerous code’ isn’t what we need saving from.

                                  Personally, I feel what we need saving from is people building software who have zero consideration for the user. So the better I can directly express mental models in software, the better the software is IMO. Modern C++ is actually really good at allowing me to say what I mean.

                                  1. 3

                                    This is based on assumptions that safety is only useful as an end to itself, and that safety decreases language’s usefulness. The counterpoint to it is that safety features eliminate entire classes of bugs, which reduces amount of time spent on debugging, and helps shipping stable and reliable programs to users.

                                    Rust also adds fearless concurrency. Thread-safety features decrease the amount of effort required to parallelize the program correctly. For example, parallel iterators are simple to use, and can guarantee their usage won’t cause memory corruption anywhere (including dependencies, 3rd party libraries!).

                                    So thanks to safety features you can solve users’ problems quickly and correctly.

                                    1. 1

                                      I feel that one day both C and C++ will be relegated to academic “teaching languages” that students will dread, that are used only to explain the history and motivations by the more complex (implementation wise) but better language that overtakes them.

                                      1. 1

                                        I am not sure why that would ever happen. As teaching languages both are pretty much useless, for the surface simplicity and hidden complexity of C or the sheer size of C++. We are currently not teaching BCPL or ABC or any other predecessors of currently popular languages, because while interesting from a historical perspective it doesn’t teach you all that much.

                                        1. 3

                                          Late response, but I totally agree with you. I was thinking of it more in terms of the way assembly is typically taught to CS students. It’s good to know that your code will be run as these machine instructions eventually, but it’s not strictly necessary for developing useful applications.

                                  2. 1

                                    Sometimes I think the whole Rust vs C++ debate is a non-starter because both camps are coming from different perspectives. At some level, I think a lot of programmers who side with C++ would prefer introducing memory/lifetime bugs at the cost of reducing performance bugs. At some level, I think a lot of programmers who side with Rust prefer introducing performance bugs at the cost of reducing memory/lifetime bugs. Unless the conversation starts with “X is the priority. Y is secondary”, it ends up going in circles.