I have a little class called EImpl that is kind of like std::indirect except that it embeds the impl instead of pointing to it. It takes three template parameters: an embedded struct, a size and an alignment. It static_asserts that the embedded struct fits in the size and alignment, and it embeds it with approximately zero overhead. It’s about as easy to use as any other pImpl technique.
But if the pimpl size grows too big, you're forced to break ABI?
And before it grows too big, it wastes memory. For your use cases it may not matter, and the saved pointer indirection may be more important, but maybe the person who has a million item vector of objects doesn't appreciate a 300% "just in case" memory overhead. The overhead may also hurt cache hits.
If you're doing this to save the pointer indirection, you should benchmark it for every use case, since negative cache effects may dwarf that gain.
Then again, extra padding can also help performance, for some workloads (especially multi threaded read/write against a vector of objects).
So without further context, there's no way to say if your way hurts or helps. It's certainly not a general solution.
"Breaking ABI" isn't an issue unless you can't compile your code anymore. It's pathetic that C++ has been so hamstrung over ABI that we're willing to stop improving.
Sadly, you can't easily do the full pimpl idiom in C++.
The pimpl idiom is a C idiom where a header declares an opaque structure and prototypes of functions that take pointers to that structure. In C the OP example would look something like
// widget.h
typedef struct Widget_t Widget; /* opaque! */
Widget* Widget_Create(const string* pName);
Widget* Widget_Clone(Widget*);
void Widget_Destroy(Widget*);
void Widget_click(Widget*);
int Widget_clickCount(const Widget*);
const string* Widget_label(const Widget*);
// widget.c
struct Widget_t {
int clicks;
string *name;
};
// ... implementations of the functions from the .h ...
In particular, in C `Widget` directly has `clicks` and `name` as fields.
But in c++ we like to use methods on objects, and in order to do this, you need the class declaration in scope, which means your current compilation unit needs to have seen all of Widget's data members. In practice this means if you try to use "pimpl" in C++, you do something like the OP where there is a pointer to an opaque type inside your class.
However, this is not the same thing. Methods are called with a `this` pointer, which means every access to the internal structure adds a second pointer dereference. This is why this isn't the true pimpl -- it wastes an extra deref on every access.
You can get true pimpl in current C++ but it's a lot of boilerplate and heavily relies on compiler inlining. An implementation of the example from the OP: https://godbolt.org/z/6EznxeG1n . In practice this is too much work, hard to read, and so nobody does it.
For the c++ standards committee: please add an "opaque class" feature where the class can only define non-virtual method prototypes. Then the full class declaration, in the associated cpp file, could include its parent classes, actual data layout, and function implementations.
> Never null: it always holds a value, except in the moved-from state
I am wondering why C++ can't implement "non-null" unique_ptr version in the same way? As I know, that the main argument against implementing it is, that it's can't be done, since move-out unique_ptr still can be null.
Hmm. Do people use PIMPL that much (I have used it, but rarely) that we need std library support (and testing, documentation, understanding)? Just asking.
It's often used in libraries where you need to guarantee ABI compatibility. Fixing a bug or implementing a feature may require adding a new member into the class, which would change its size (thus break ABI compatibility). PIMPL is the typical solution here, since the inner/impl class is not part of the public ABI.
I also like to use it sometimes to "hide" private methods and their documentation into PIMPL, so the public header is kept clean.
This std::indirect thingie looks more like a general helper for any data 'dangling off' an object, not limited to pimpl.
Not sure how much pimpl is used in reality, but it's a pretty ok solution to speed up build times (apart from unity builds), because it avoids having to include headers that are only needed for the private state into the public interface header.
Indeed. I primarily used PIMPL when I want to avoid polluting public header files with implementation detail #includes in cases where forward declarations are impossible or unwieldy and inline methods are irrelevant.
My approach to reducing the compile time of code which uses a class is moving the functionality out of the class and into standalone functions; or at least moving the method definitions into a non-header `.cpp` file.
I remember using it all the time for the Windows headers because they pollutes the compilation unit like you wouldn't believe — the rule was to only include them in c/cpp files.
The irony is that including/using many standard c++ headers is far far more expensive than including a lean windows.h these days.
To make hobby-coding fun, i use a mstdp.hpp that implements "naive" versions of unique,shared,function,etc that compiles faster than including just one of the std versions (and yes, MSVC versions of those libraries seem to be excessivly complex).
Where are we with modules, isn't pimpl there largely to avoid costs related to including the world?
I was pondering on why he was putting the defaulted methods in the cpp, any particular reasons?
I did realize that the indirect version is required to be in the cpp since the header won't know how to copy without knowing the definition of the impl class.
This looks great indeed - I wonder if there are any particular gotchas, though, as things often are in C++next land.
With many of the features coming into the language over time, I kinda wish that a bit more restricted subset of it eventually becomes a thing, but I know in practice it might as well be a completely different language. That, and I expect that still many other things have not been resolved as well as they are elsewhere, such as build system and dependency management (although I haven't touched this stack for a while now, so I would love to be surprised).
1. click() should not be a member of the widget. A widget does not click; a user clicks a widget. A click can change a widget's state, but the state might change because of other effects, e.g. pressing a key when the widget is focused. But then, that's just one of the issues with treating UI widgets this way.
2. More to the point - clickCount. If this is a button, it shouldn't keep a record, or aggregate, of its clicks within it; and if it's a widget where this does really matter, like a range control where more clicks mean a value that goes farther along the range - you still would not keep the count of clicks, but the current position. Statistics about the interaction with an object should not be part of the object itself. At most it might be legitimate to have, say, a Widget class, a template like <class Stats> StatisticsTracker , and then class TrackedWidget which uses that as a mixin, i.e. inheriting both Widget and StatisticsTracker<ClickStats>. And that's already stretching it beyond what I would find reasonable.
3. Having something named is another aspect of objects which may be a good fit for a mixin class.
Anyway, an 'indirect' type for objects you don't know the definition of sounds nice.
A few more nitpickis about the example:
1. Instead of explicitly applying the rule-of-0 with `= default` for the copy&move ctor&assignment and the destructor - just _don't_ write anything:
class Widget
{
public:
void click();
int clickCount() const;
std::string label() const;
private:
struct Impl;
std::indirect<Impl> pimpl_;
};
and that's the beauty of the rule of 0.
2. Why return an std::string for the label? The label() method should return an std::string_view
This is just a simple example, therefore nitpicking on the semantics of the Widget methods is a bit silly.
> 1. Instead of explicitly applying the rule-of-0 with `= default` for the copy&move ctor&assignment and the destructor - just _don't_ write anything:
The blog post explicitly explains why this doesn't work. You have to define these methods in the source file because they need to see the definition of the Impl struct.
C++ is getting more and more complex. It used to be said that people use only a small percentage of it when writing C++, but I am beginning to think that the cake is a lie here.
Besides being a common idiom, for how many warts C++ might have, no one is rewriting LLVM, GCC, V8, JVM/ART and .NET runtimes, CUDA/Metal/DirectX, Unreal, Godot,.... into something else, RIR is not happening there.
People will contend themselves with "C++ the good parts", helped by clang-tidy, PVS, MSVC analyse, and move on.
Is it actually simpler, though? The unfortunate reality of this world is the fact that C++ is not the latest standard of the language or the newest shiny library, it's all of them at the same time. Adding a new way of doing the same thing decreases complexity only if you migrate all of the existing code, which nobody ever does.
I think the parent's point is that we started with raw pointers to implement PIMPL, then we had std::unique_ptr, and now we have std::indirect. So there are now three different ways how PIMPL can be implemented, each has its gotcha's and subtle differences that one needs to keep in mind. In large codebases you will now have to deal with all three solutions being used, depending on how old the code is.
The point of each improvement is fewer easily-made errors. Having implicit deep copying handled avoids lots of errors with manually implementing it the oldest way.
Well that's a lie. I've long been back to raw pointers and it's by far the easiest way to do it. All of Pimpl, unique_ptr, and whatever other clever mechanism (I'm not even looking at std::indirect anymore) just aren't really ergonomic.
Nobody needs "deep copying", ever. It's not even well defined what it should mean (i.e. how deep etc.). It's purely a theoretical problem with no good practical (one-fits-all) solution. The only practical way is to copy what you need copied, when you need it. Done.
std::indirect looks for me like another pointless c++ thing that already works with forward pointer declaration. You can add it to another ton of pointless things C++ adds without fixing the old ones. The issue with c++ is that it is so big, that everyone uses some kind of dialect of it and the fancier it gets, the less readable it becomes and the more magic happens behind the curtains.
A developer of a C++ codebase now has to learn a specific meta language of this codebase. Fuck that, I have enough languages and their idiosynchronies to remember for my work now. After using Go for a pair of years returning to C++ is like coming back to a big archaic mess. I'll just go learn Rust instead and forget all those new useless C++ templates like std::indirect
I think it's kind of awkward either way. The standard committee keeps adding new features to the language to address common pain points in the industry. But many people don't have that much time to learn the new features, and hates it when seeing something in the code but can't intuitively understand what it's doing. I once witnessed a 10+ year C++ coder (that had been immersed in some old C++ code base for many years) seeing a piece of C++14 code for the first time -- he said it reads like an entirely different language, not the C++ he's familiar with at all.
> But many people don't have that much time to learn the new features
Because they spend so much of their time struggling with the pain points of the older code.
> but can't intuitively understand what it's doing
For (most?) new vocabulary types, it is rather intuitive to understand what they do. optional, variant, indirect - you may not remember the details by heart immediately, but you get the general idea and expect that they would behave in some reasonable way. And mostly, they do. That's not to say they're perfect: I feel like vomiting looking at std::variant's and how you have to work with them, as opposed to a proper case classes / algebraic union types in the language itself. And yet - when someone puts one in their class, instead of a bunch of code in a bunch of methods, you know what's going on. It does "read like a different language" somewhat, and that's good. The nicer language has been struggling to get out, as the saying goes.
Pretty much all the C++ features I've used are good enough to write some toy code snippet, but it's hard to use them to good effect at scale without causing massive problems.
Even classes are an instance of this, they were to solve some perceived problems, but they created much bigger issues, such as readability issues and introducing many more compile time dependencies.
PIMPL wasn't even a C++ feature but an idiom pushed by some people. It is next to unusable because you have to duplicate the API and write all the call forwards.
One problem with std::unique_ptr for example is that there is no ergonomic way to use it to hide implementations. The reason is it relies on destructors and to use destructors the class definition needs to be visible.
This is actually useful, but despite it is another extra thing you will have to remember when reading C++ code. I guess with LLMs things aren't so bad.
Why? It’s still the good (bad)
old pimpl pattern. It just got a bit shorter. When reading you dont even need to grok “std::indirect”, you see the word pimpl and you know what’s going on.
I don't really get why people keep repeating the "C++ is too big" complaint together with the implication that you need to remember the entirety of the standard library. In comparison Java has networking, GUI framework and even MIDI in its standard libraries. Is it because C++ is more closely related to C which library is so small that it barely contains anything useful? I much prefer code that uses a library feature rather than yet another poorly implemented and not documented hand rolled version of it.
Networking, GUI frameworks, and MIDI are presumably all self-contained and you would not need to be familiar with them except when working on networking, GUIs, or MIDI files, respectively. This is a general-purpose thing that could show up in any c++ code.
You need to remember _less_, rather than more, when you use this kind of vocabulary types. Think about std::optional. Before that (and if you didn't write something like it yourself), you had to, for each class, remember the bespoke semantics of when and how it represents the lack of some members, and you would have to have non-defaulted ctors, move assignments and dtors, and then whenever you used that class you would need to think about what those custom method do, which might be different than other classes which have optional members. Now you just tell yourself "oh, it just has an optional member, no biggie". Look at my comment above regarding how short the implementation of Widget becomes when you squeeze the juice from having the rule of 0.
I thought C++ is unnecessarily complex, and then I see Rust following the same pattern... I've just thought of a complexity metric that would calculate the ratio of alphanumeric characters to punctuation.
And before it grows too big, it wastes memory. For your use cases it may not matter, and the saved pointer indirection may be more important, but maybe the person who has a million item vector of objects doesn't appreciate a 300% "just in case" memory overhead. The overhead may also hurt cache hits.
If you're doing this to save the pointer indirection, you should benchmark it for every use case, since negative cache effects may dwarf that gain.
Then again, extra padding can also help performance, for some workloads (especially multi threaded read/write against a vector of objects).
So without further context, there's no way to say if your way hurts or helps. It's certainly not a general solution.
The pimpl idiom is a C idiom where a header declares an opaque structure and prototypes of functions that take pointers to that structure. In C the OP example would look something like
In particular, in C `Widget` directly has `clicks` and `name` as fields.But in c++ we like to use methods on objects, and in order to do this, you need the class declaration in scope, which means your current compilation unit needs to have seen all of Widget's data members. In practice this means if you try to use "pimpl" in C++, you do something like the OP where there is a pointer to an opaque type inside your class.
However, this is not the same thing. Methods are called with a `this` pointer, which means every access to the internal structure adds a second pointer dereference. This is why this isn't the true pimpl -- it wastes an extra deref on every access.
You can get true pimpl in current C++ but it's a lot of boilerplate and heavily relies on compiler inlining. An implementation of the example from the OP: https://godbolt.org/z/6EznxeG1n . In practice this is too much work, hard to read, and so nobody does it.
For the c++ standards committee: please add an "opaque class" feature where the class can only define non-virtual method prototypes. Then the full class declaration, in the associated cpp file, could include its parent classes, actual data layout, and function implementations.
I am wondering why C++ can't implement "non-null" unique_ptr version in the same way? As I know, that the main argument against implementing it is, that it's can't be done, since move-out unique_ptr still can be null.
https://github.com/microsoft/GSL/blob/main/docs/headers.md#u...
What do you mean by move-out unique_ptr? That the not_null ptr type would ne null after it's been moved?
In that case that's just a plain usage error, same as how you could memset it to null.
I also like to use it sometimes to "hide" private methods and their documentation into PIMPL, so the public header is kept clean.
Yep, that's what I've used it for. Didn't find it too difficult to implement it myself, but I guess every bit of convenience/bug avoidance helps.
Not sure how much pimpl is used in reality, but it's a pretty ok solution to speed up build times (apart from unity builds), because it avoids having to include headers that are only needed for the private state into the public interface header.
To make hobby-coding fun, i use a mstdp.hpp that implements "naive" versions of unique,shared,function,etc that compiles faster than including just one of the std versions (and yes, MSVC versions of those libraries seem to be excessivly complex).
I was pondering on why he was putting the defaulted methods in the cpp, any particular reasons?
I did realize that the indirect version is required to be in the cpp since the header won't know how to copy without knowing the definition of the impl class.
With many of the features coming into the language over time, I kinda wish that a bit more restricted subset of it eventually becomes a thing, but I know in practice it might as well be a completely different language. That, and I expect that still many other things have not been resolved as well as they are elsewhere, such as build system and dependency management (although I haven't touched this stack for a while now, so I would love to be surprised).
All this complexity follows unique_ptr and copy constructor madness.
Anything with pointers with ownership should never be copied - period. Reference pointers - OK if scope/lifetime is known.
Can we have c++11 lite?
1. click() should not be a member of the widget. A widget does not click; a user clicks a widget. A click can change a widget's state, but the state might change because of other effects, e.g. pressing a key when the widget is focused. But then, that's just one of the issues with treating UI widgets this way.
2. More to the point - clickCount. If this is a button, it shouldn't keep a record, or aggregate, of its clicks within it; and if it's a widget where this does really matter, like a range control where more clicks mean a value that goes farther along the range - you still would not keep the count of clicks, but the current position. Statistics about the interaction with an object should not be part of the object itself. At most it might be legitimate to have, say, a Widget class, a template like <class Stats> StatisticsTracker , and then class TrackedWidget which uses that as a mixin, i.e. inheriting both Widget and StatisticsTracker<ClickStats>. And that's already stretching it beyond what I would find reasonable.
3. Having something named is another aspect of objects which may be a good fit for a mixin class.
Anyway, an 'indirect' type for objects you don't know the definition of sounds nice.
A few more nitpickis about the example:
1. Instead of explicitly applying the rule-of-0 with `= default` for the copy&move ctor&assignment and the destructor - just _don't_ write anything:
and that's the beauty of the rule of 0.2. Why return an std::string for the label? The label() method should return an std::string_view
> 1. Instead of explicitly applying the rule-of-0 with `= default` for the copy&move ctor&assignment and the destructor - just _don't_ write anything:
The blog post explicitly explains why this doesn't work. You have to define these methods in the source file because they need to see the definition of the Impl struct.
This only works if it's always the same value. This doesn't work if the label is for example, set to `std::to_string(clickCount())`
People will contend themselves with "C++ the good parts", helped by clang-tidy, PVS, MSVC analyse, and move on.
This way I have already replaced a lot of for-loops by range-based for-loops. It helps me to understand code faster.
But code parts that noone needs to touch or see do not need to be more readable.
Nobody needs "deep copying", ever. It's not even well defined what it should mean (i.e. how deep etc.). It's purely a theoretical problem with no good practical (one-fits-all) solution. The only practical way is to copy what you need copied, when you need it. Done.
Because they spend so much of their time struggling with the pain points of the older code.
> but can't intuitively understand what it's doing
For (most?) new vocabulary types, it is rather intuitive to understand what they do. optional, variant, indirect - you may not remember the details by heart immediately, but you get the general idea and expect that they would behave in some reasonable way. And mostly, they do. That's not to say they're perfect: I feel like vomiting looking at std::variant's and how you have to work with them, as opposed to a proper case classes / algebraic union types in the language itself. And yet - when someone puts one in their class, instead of a bunch of code in a bunch of methods, you know what's going on. It does "read like a different language" somewhat, and that's good. The nicer language has been struggling to get out, as the saying goes.
Even classes are an instance of this, they were to solve some perceived problems, but they created much bigger issues, such as readability issues and introducing many more compile time dependencies.
PIMPL wasn't even a C++ feature but an idiom pushed by some people. It is next to unusable because you have to duplicate the API and write all the call forwards.
One problem with std::unique_ptr for example is that there is no ergonomic way to use it to hide implementations. The reason is it relies on destructors and to use destructors the class definition needs to be visible.