This is kind of a tricky thing to really handle correctly.
In practice, most people run with memory overcommit turned on. So you never really get std::bad_alloc or a NULL pointer back from functions which allocate memory. Instead, the OOM killer just starts killing once the memory usage gets high enough.
You may see std::bad_alloc or a NULL return if you request a ridiculuously large amount of memory, like with
char* foo = malloc(-1);
But that's just the system's way of saying, "well, that input parameter makes no sense."
In cases where people do run with overcommit disabled (or use rlimit), you can actually get NULL returns or std::bad_alloc. But then how do you handle it? If you try to do anything with a std::ostream, you're calling malloc again, which is not likely to succeed. Maybe the best thing to do is to augment the logging code with something that just calls write(2) on some file descriptors. If you directly invoke that, you might actually succeed in getting something into the logfile / stdout / stderr / syslog / wherever. Oh, also note that glibc's version of syslog() sometimes calls malloc(). So that one should definitely be called last.