Friday, August 12, 2011

The 64-bit Observation Counter

The EXTENDOBSCOUNTER option is mentioned in the announcement for the new edition of Professional SAS Programmer’s Pocket Reference, and that made me realize I needed to write a post to explain what it is.

It is no longer a gee-whiz moment when a data table exceeds 2 billion rows. And with the computers of 2011, it is not such a big deal to be processing that many observations in a SAS program. So within a few years, it would have started to seem quaint that SAS can count observations only up to about 2 billion.

That is why SAS set its sights on a new observation counter that can count observations up to 9 quintillion. The new 64-bit observation counter is available in SAS 9.3, but it is not the default because it is not compatible with any prior SAS release. If you are working with data that could run into the billions of observations, though, you would do well to start using the new expanded observation counter as soon as you complete the transition from SAS 9.2 to SAS 9.3.

All this requires is the EXTENDOBSCOUNTER=YES option in the LIBNAME statement for the library. To use a 64-bit observation counter on an individual SAS data file, write the option as a data set option when you create the file.

The 64-bit observation counter will become the norm, I am sure, at some point in the future when the chances of encountering SAS 9.2 and earlier SAS releases become relatively slight. At this point, though, it is important to note that the EXTENDOBSCOUNTER=YES option makes SAS data sets incompatible with SAS 9.2, so it isn't the right move in a company that is still using SAS 9.2 on some machines. At worst, it would force you to make copies of SAS libraries in SAS 9.3 before you could use them in SAS 9.2 — not a big deal if there are just a few million observations, but a process you would rather avoid if the observation count is closer to a billion.

No comments:

Post a Comment