Buffering Number Ranges and Legal Framework
*Buffering Number Ranges and Legal Framework *
* *
*Buffered accesses *
In order to improve performance, several buffering options have been devised, therefore, when investigating an issue concerning number ranges it is not enough to state “number range object is buffered” but to answer the question “how ?”
The buffering of number ranges is set, along with other attributes of the number range objects, via transaction SNRO. In the number range objects attributes, depending on release, the buffering is shown either in the sole page displayed for the object, or in later releases in the customizing tab. Along with the buffering type used, the buffer size is also displayed.
A BUFFER SIZE OF ZERO is the same as not buffered, regardless of the buffering type displayed, regardless of the buffering method used.
h5. *How do I know, which is my object? *
Many number range objects have their own number range transaction, therefore the object name does not appear.
As an example for the determination of the number range object, let’s take the well-known transaction for financial documents, FBN1. You go to FBN1 and start the debugger ( /h in the ok-code and hit enter). Upon entering the debugger, you should display the value of the field TNRO-OBJECT. This the name of the number range object, in this case RF_BELEG,
Or transaction SE16, table TNRO, field CODE = FBN1.
From time to time, due to locks in the table NRIV, dumps occur with timeout, in the module SAPLSNR3, at the instruction
SELECT SINGLE FOR UPDATE * FROM NRIV
If you want to determine in the dump the name of the object, you will find it in the field P_OBJECT. Furthermore, the dump tells you which type of buffering is used, in the field TNRO-BUFFER.
The corresponding values (and the possible types of buffering) are:
SPACE no buffering
X main memory buffering
L local buffering (obsolete)
P extended local buffering
S parallel buffering.
Observation: you may distinguish between number range objects on one side and intervals on the other side. There may be one or more intervals for the same object.Buffering is set at object level for all intervals alike. There may be variations though, like except specific intervals from buffering.
h5. *First considerations of the legal aspects of numbering *
Many number ranges have no legal constraints, meaning the numeration does not need to be neither ascending nor gapless, allowing for buffering techniques to be used in order to improve performance.
An example is the spool number of a document. It does not matter if a number given later is lower than a number given earlier, AS LONG AS THE NUMBERS REMAIN UNIQUE.
Nearly all number range objects are delivered in the SAP standard as buffered in main memory.
Main memory buffering is the most efficient in terms of performance and has two side effects:
– the numbers are not given in ascending order
– the numbering may have gaps.
An interesting aspect is that some customers, conscious of the connection between buffering and the way the numbers are allocated, decide upon de-buffering the range because they do not LIKE these side effects, not because there is some law against it.
Other customers are complaining about these side effects considering them a bug.
But it is not, it works as designed.
De-buffering a number range object, delivered by SAP as buffered in main memory can (must not) be extremely damaging to the system performance in the specific application area and is regarded by SAP as a customer modification.
(see note 678501 on this subject).
h5. *Unbuffered number ranges *
On the other hand, there ARE number range objects which are subject in some countries to legal restrictions, like FI documents (object RF_BELEG) and SD (RV_BELEG). Such objects are delivered by SAP always as unbuffered, further attempts to improve performance by buffering being subject to laws specific to the country where the customer operates, or more complex when the customer is operating in a multi-national environment with different legal approaches to the issue.
In order to access the landscape of legal limitations across different countries and regions in this area SAP started an operation of evaluation, but it had to be stopped, not only because of the complexity of the issue but also because its volatility.
For example, until 2006, in Spain the financial documents and the order numbers had to be gapless but not necessarily in ascending order. Since 2006 a new law was passed, stating it has to be both.
For the time being, in Germany it has only to be gapless. In Israel, the numbering has to be ascending, but gaps are allowed.
The conclusion is, it is the customer who has to provide the information about the legal framework and SAP to make recommendations accordingly. In order to make the appropriate recommendation, one should be aware which effects which type of buffering (and buffer size) have and which combinations are possible.
Some applications as FI and SD allow a bigger flexibility in this respect, via specific customer exits, which can influence the buffering according to the demands of the application.
Let’s take a look at different options of the buffer and the corresponding results on the numbering from the legal standpoint also.
On the other hand, there ARE number range objects which are subject in some countries to legal restrictions, like FI documents (object RF_BELEG) and SD (RV_BELEG). Such objects are delivered by SAP always as unbuffered, further attempts to improve performance by buffering being subject to laws specific to the country where the customer operates, or more complex when the customer is operating in a multi-national environment with different legal approaches to the issue.
In order to access the landscape of legal limitations across different countries and regions in this area SAP started an operation of evaluation, but it had to be stopped, not only because of the complexity of the issue but also because its volatility.
For example, until 2006, in Spain the financial documents and the order numbers had to be gapless but not necessarily in ascending order. Since 2006 a new law was passed, stating it has to be both.
For the time being, in Germany it has only to be gapless. In Israel, the numbering has to be ascending, but gaps are allowed.
The conclusion is, it is the customer who has to provide the information about the legal framework and SAP to make recommendations accordingly. In order to make the appropriate recommendation, one should be aware which effects which type of buffering (and buffer size) have and which combinations are possible.
Some applications as FI and SD allow a bigger flexibility in this respect, via specific customer exits, which can influence the buffering according to the demands of the application.
Let’s take a look at different options of the buffer and the corresponding results on the numbering from the legal standpoint also.
h5. *No buffering *
In this I quote from the excellent FI note 1398444:
The document number is assigned in chronologically ascending orderbased on the table NRIV. For this, the table NRIV is locked until the application (LUW) is terminated by either a COMMIT WORK or ROLLBACK (see also Note 639754). Another application cannot take a document number during this time.
The update is called by the COMMIT WORK and the document number is assigned permanently. In the case of a rollback, the document number that was just used is not assigned and is available for the next posting in NRIV again.
This lock guarantees a chronologically ascending assignment of the document number without gaps. However, the lock causes a serialization in the table NRIV, which can seriously impair the system performance (see also Note 678501). There are performance problems in particular in the case of parallel batch processes, as the lock is held for a very long time.
h5. *Main storage buffering *
On demand – which means when a number is asked for the first time since restart of a certain SAP instance – a set of numbers, as many as the buffer size is set – are downloaded in main memory from the respective interval in the table NRIV. The interval level in NRIV is increased by the same amount.
This happens through a “back door” process and does not create locks. When the instance is restarted again, these numbers are lost. Is the system a multi-instance system – as most are – in time, each instance receives its set of numbers and the applications running here or there will get numbers from the respective set. The result is that numbers for the application are jumping to and fro.
If a rollback happens, the number drawn is lost forever. If the system is restarted, the buffer is deleted and the numbers there are lost. Main memory buffering works roughly in the following way:
A separate process downloads – as aid, on demand – at SAP instance level – a group of numbers of the size given in the buffer size in the transaction SNRO – in the main memory of the particular instance and gives one or more of them to the application, the rest remaining in the main memory buffer for later use.
This separate process issues an immediate commit work for that process, leaving the NRIV interval free for further accesses. The whole process is done and controlled by the SAP Task Handler and the process of reloading can be seen in SM50 as a very short process with the user SAPSYS.
When everything runs normally, main memory buffering ensures a high-performance handling of the number ranges without any problems.
Nevertheless problems may appear. The most common comes from the reload in a parallel process which uses a DIA work process type. It may happen, that due to an extreme load of the instance by other processes, there is simply no DIA work process available anymore. For this, consequences and solutions see the note 920234.
There is also the aspect of the maximum number of buffer entries. The functioning can be impaired by the fact that a new entry in the buffer cannot be created, and the associate error is NR031. The maximum size and the actual use, as well as the content can be monitored with transaction SM56 and the relevant note is 399207.
h5. *Now, how efficient is this type of buffering? *
The efficiency has two aspects:
First, the fact that there is no lock on NRIV between the number drawing and a commit work. This is enormous.
The second aspect is the size of the buffer. In theory, the bigger the buffer, less “back door” processes are needed to refill the buffer. But this is a small matter, except for installations where there is a very low number of available dialogue work processes; a smaller buffer would create a surplus strain on the systems which may create problems. On the other hand, the difference between buffered and unbuffered access is so much more important, that a new idea should be considered here.
h5. *What happens if the buffer size is set to exactly 1? *
Then each number draw will happen through the “back door”, still from NRIV, but delivering the number without any lock. The disadvantage is, in case of a rollback, the number is lost, on the other side, REGARDLESS OF THE INSTANCE where it is running, the numbers will be delivered in an ascending sequence.
Conclusion: Buffering in main memory with a buffer size of 1 ensures an ascending sequence, but cannot avoid gaps in case of rollbacks.
h5. Local buffering (obsolete)
It was the first try to get some type of buffering for documents where it is important to have no gaps, even if documents are not allocated in an ascending order.
This buffering method was the first attempt at SAP to solve the issue of gaps in case of a rollback or a system restart. It does the same thing as the main storage buffering, but instead of main memory use where numbers are lost in a restart the table NRIV_LOKAL is used instead; there is no “hidden hand” to reload the buffers (meaning when reloading and another instance wants reloading they run to the same lock in NRIV as if unbuffered) and respectively processes running in the same instance and wanting numbers from the same set are locking themselves on NRIV_LOKAL instead of NRIV; maybe a bit later than unbuffered but still unavoidable.
This buffering method is not eligible online anymore in later releases and considered by SAP as completely obsolete. This buffering method is on the verge of becoming obsolete.
h5. *Parallel buffering *
This buffering method is simply an enhancement of the extended local buffering, introducing the “back door” mechanism in order to refill the buffers – always at work process level – but keeping the numbers reusable in case a rollback takes place, just like the extended local buffering does.
It uses for storing the buffers the table NRIVSHADOW, and not NRIV_LOKAL as the extended local buffering does.
Due to the back door mechanism, there are no locks set on the table NRIV when using it, and therefore the buffer size can be considerably reduced, even to a value of 1 and still with considerable performance increase as compared with the unbuffered access.
h5. *What is the result of parallel buffering with a buffer size of 1? *
– the numbers are drawn in ascending order but without locks on NRIV, as long as there is no ROLLBACK happening.
– when a rollback happens, the number returns to the buffer, meaning that it may be assigned at a much later time, out of sequence, but this is not the normal case, as rollbacks seldom happen
– meanwhile these numbers can be documented for the audit with a report which shows which numbers are still available BELOW the highest document number allocated.
If the buffer size is bigger than 1, it works externally exactly like the extended local buffering, but internally its performance is much better.
This method is recommended to be used instead of extended local buffering whenever legally acceptable, and in view of what was said above, either with a buffer size of 1 or bigger.
It is implemented for SAP releases >= 4.6C and has an important technical improvement starting with basis release 620.
Prior to this technical improvement, the reload of the buffer happened just as in the case of main memory buffering, in a separate DIA work process.
Since Rel. 620 and note 1043348, no separate work process is needed, because the reload uses a separate database connection.
h5. *Why use separate processes or connections? *
The programs run by SAP application have to take care of data integrity in a business process, for instance a billing should either finish completely or not take place at all. This is driven by the application with the use of COMMIT WORK or respectively ROLLBACK WORK. Therefore, if the number range loading of buffers has to issue a COMMIT WORK, it cannot do it in the same process as the application asking for numbers, else it would commit the process data prematurely.
h5. *Parallel buffering – the “italian” solution. *
We call it internally “Italian” because the idea appeared in connection with an Italian customer.
The main problem is that, even with a buffer size of 1, depending on how many number have been drawn between the first number and COMMIT or ROLLBACK, if a rollback occurs, one or more numbers are returned to the buffer for further use. This may happen the same day, but also many
days later. These numbers would then be out of sequence which is contrary to the laws in Italy and several other countries.
Therefore we devised a solution which is both buffered and compliant to this part of the law.
The numbers returned to the buffer will not be used later, but parked in a special documentation table called NRIV_RESTE and the application receives a brand new number.
Two questions arise, a minor and a major one concerning this logic.
The minor question: when do the “odd” numbers get promoted to NRIV_RESTE?
The answer is: when the number should have been drawn out of sequence, not earlier. Why then not do it at the proper moment, when the ROLLBACK occurs? Because you cannot write anything to any database while a ROLLBACK takes place. But the documentation program RSSNR0S1 shows both the buffer content and the content of NRIV_RESTE, so the moment of the move plays no role.
The major question: if the numbers which are rolled back are not used anymore, does then appear a gap in the numbering?
The answer is, yes, but.
At this point it is appropriate to remark that gaps may always occur when there is some type of unusual accident, like an interruption of the update process. Rollbacks are also unusual occurrences in the type of programs relevant for this issue and due to some type of error. This means in the normal process gaps never occur. This may not be easy to be accepted by financial audits, but it is the only compromise way between legality and performance.
In order to use this solution, some customizing is necessary including the activation and specific coding of customer exits.
The relevant note is 840901, then notes specific for FI and SD where such exits exist which are also mentioned in the note 840901.
Another side effect of the buffering is the way processes in parallel are getting the numbers. Suppose there are two background processes drawing numbers from the same interval. Without using the buffering, they will have to wait for one another so that a process will always get numbers in sequence while holding the interval in exclusivity. With parallel buffering in place, the application table, where these numbers are used, will have no gaps, but each process helps itself to numbers without waiting, each time it needs one, so the numbers received will not be in gapless sequential order. This is of no consequence for normal processing, but is significant for the next chapter, legal aspects of the electronic signature in Portugal.
h5. *Special case Portugal 2011: Electronic signature *
This part should be read only by consultants who have to do with electronic signature for invoices in Portugal.
The Portuguese rules for electronic signature ask for using the previous invoice number, which has to be gapless, to generate the signature for the actual invoice. This contradicts completely any attempt at buffering in this area.
This problem can be prevented by generating the signatures at a later moment, when all numbers have been allocated; this has already been addressed and the solution exist. It is though not practical as many customers decide to print the invoices in the very moment of their creation.
The conclusion is that at this moment buffering for invoice numbers cannot be used in Portugal.
New NetWeaver Information at SAP.com
Very Helpfull