ORACLE 12.2 – deprecated pragma for PL/SQL units

Tags

, , , ,

With ORACLE 12.2 it is possible to mark program units as deprecated along with a message using a new pragma. The message will show up when the deprecated program unit is referenced and the referencing program unit is compiled (and compiler warnings are enabled).

This enables us to gently replace code by advising the user to move to an alternative program unit.

 


EXAMPLE: Standalond procedure

ALTER SESSION SET PLSQL_WARNINGS = 'Enable:All'
/

CREATE OR REPLACE PROCEDURE p IS
   PRAGMA deprecate(p,'p is deprecated please use p2 instead.');
BEGIN
   sys.dbms_output.put_line('p');
END p;
/

Warning(2,4): PLW-06019: entity P is deprecated

When compiling a deprecated program unit we get an PLW-06019 warning.


EXAMPLE: Packaged procedure

CREATE OR REPLACE PACKAGE pkg IS
   PROCEDURE p;
     PRAGMA deprecate(p,'pkg.p is deprecated please use pkg.p2 instead.');
 
   PROCEDURE p2;
END;
/

Warning(2,4): PLW-06019: entity P is deprecated

CREATE OR REPLACE PACKAGE BODY pkg IS
   PROCEDURE p IS
   BEGIN
      sys.dbms_output.put_line('pkg.p');
   END p;

   PROCEDURE p2 IS
   BEGIN
      sys.dbms_output.put_line('pkg.p2');
   END p2;
END pkg;
/

The pragma specification for packaged program units has to be placed in the specification.

 


Referencing a deprecated program unit

So, what happens if you are using pkg.p or p?

CREATE OR REPLACE PROCEDURE q IS
BEGIN
   p();
   sys.dbms_output.put_line('q');
 
   pkg.p();
   sys.dbms_output.put_line('q'); 
END q;
/

Warning(3,4): PLW-06020: reference to a deprecated entity: p is deprecated please use p2 instead.
Warning(6,8): PLW-06020: reference to a deprecated entity: pkg.p is deprecated please use pkg.p2 instead.

A different warning (PLW-06020 instead of PLW-06019) shows up including the message we placed in the pragma.


Forcing PLW-06020 to be an error

If we would like to force the user of pkg.p/p to move to pkg.p2/p2 we could now easily defined PLW-06020 to be a compiler error.

ALTER SESSION SET PLSQL_WARNINGS = 'ERROR:6020'
/

CREATE OR REPLACE PROCEDURE q IS
BEGIN
   p();
   sys.dbms_output.put_line('q');
 
   pkg.p();
   sys.dbms_output.put_line('q'); 
END q;
/

Error(8,4): PLS-06020: reference to a deprecated entity: p is deprecated please use p2 instead.
Error(11,8): PLS-06020: reference to a deprecated entity: pkg.p is deprecated please use pkg.p2 instead.

So the q procedure does not compile anymore.


Conclusion

The deprecated pragma is a good way to let others know that a program unit should not be used anymore…if:

  • compiler warnings are enable
  • compiler warnings are looked at

The possibility to turn a warning into an error is also helpful to force the replacement of deprecated program units (in dev environment).

The problem I see is, that not all deprecated program units have the same expiry date and therefor setting a warning type to error for this case is somehow not sufficient. What I really would like to have is a third (optional) parameter in the pragma specification where I could specify by which date THIS deprecation turns into an error.

ORACLE 12.2 – Data conversion enhancements – 2

Tags

, , ,

A new SQL function in ORACLE 12.2 is validate_conversion which enables us to check whether a conversion is possible or not receiving a “boolean” (0/1) value as result.

 


Example checking conversion of a string value into a date

In case of a string to date conversion error we would like to return the 1st of January of 2016.

WITH data (str) AS (SELECT '10.15.2015' FROM dual union all
                    SELECT '15.10.2015' FROM dual union all
                    SELECT '30.02.2015' FROM dual
                   )
SELECT str
     , VALIDATE_CONVERSION(str as date,'DD.MM.YYYY') conversion_possible
 FROM data
/

STR        CONVERSION_POSSIBLE
---------- -------------------
10.15.2015                   0
15.10.2015                   1
30.02.2015                   0  

When checking conversion of a string into a date it is also possible to specifcy the language of the date string.

WITH data (str) AS (SELECT '15. March 2015' FROM dual union all
                    SELECT '15. März 2015' FROM dual union all
                    SELECT '15. Mars 2015' FROM dual
                   )
SELECT str
     , VALIDATE_CONVERSION(str as date,'DD. Month YYYY','NLS_DATE_LANGUAGE=GERMAN') conversion_possible
  FROM data
/

STR            CONVERSION_POSSIBLE
-------------- -------------------
15. March 2015                   0
15. März 2015                    1
15. Mars 2015                    0


Conclusion

Another great little helper especially when  loading data from 3rd parties. How often have we struggled just because a single value in a single record just did not fit the specifications. Using validate_conversion we can easily find those values, exclude them, log them and so on.   

ORACLE 12.2 -Data conversion enhancements

Tags

, , , , , ,

ORACLE 12.2 has a new option to handle conversion errors when using functions like CAST, TO_NUMBER, TO_CHAR, TO_DATE, etc. Using this option we may avoid conversion errors an return a default value instead.

The examples use TO_DATE but the option is useable in the other conversion functions accordingly


EXAMPLE TO_DATE

In case of a string to date conversion error we would like to return the 1st of January of 2016.

WITH data (str) AS (SELECT '15.01.2016' FROM dual UNION ALL
                    SELECT '30.06.2016' FROM dual UNION ALL
                    SELECT '30.02.2016' FROM dual UNION ALL
                    SELECT 'gugus' FROM dual UNION ALL
                    SELECT '07.03.2016' FROM dual UNION ALL
                    SELECT '15.10.2016' FROM dual)
SELECT str
 , TO_DATE(str DEFAULT '01.01.2016' ON CONVERSION ERROR,'DD.MM.YYYY') AS conversion
 FROM data
/

STR CONVERTED 
---------- ----------
15.01.2016 15.01.2016
30.06.2016 30.06.2016
30.02.2016 01.01.2016
gugus      01.01.2016
07.03.2016 07.03.2016
15.10.2016 15.10.2016

6 rows selected.  

Using SYSDATE as default

Using SYSDATE as the default value instead of a literal value works as expected.

WITH data (str) AS (SELECT '15.01.2016' FROM dual UNION ALL
                    SELECT '30.06.2016' FROM dual UNION ALL
                    SELECT '30.02.2016' FROM dual UNION ALL
                    SELECT 'gugus' FROM dual UNION ALL
                    SELECT '07.03.2016' FROM dual UNION ALL
                    SELECT '15.10.2016' FROM dual)
SELECT str
 , TO_DATE(str DEFAULT SYSDATE ON CONVERSION ERROR,'DD.MM.YYYY') AS converted
 FROM data
/ 

STR CONVERTED 
---------- ----------
15.01.2016 15.01.2016
30.06.2016 30.06.2016
30.02.2016 30.09.2016
gugus      30.09.2016
07.03.2016 07.03.2016
15.10.2016 15.10.2016

6 rows selected. 

Default to start of the year…

So next step would be to invoke a function on sysdate, to get the first day of the current year as the default value for an occuring conversion error.

WITH data (str) AS (SELECT '15.01.2016' FROM dual UNION ALL
                    SELECT '30.06.2016' FROM dual UNION ALL
                    SELECT '30.02.2016' FROM dual UNION ALL
                    SELECT 'gugus' FROM dual UNION ALL
                    SELECT '07.03.2016' FROM dual UNION ALL
                    SELECT '15.10.2016' FROM dual)
 SELECT str
      , TO_DATE(str DEFAULT TRUNC(SYSDATE,'YEAR') ON CONVERSION ERROR,'DD.MM.YYYY') AS converted
 FROM data
/ 

Error at Command Line : 8 Column : 28
 Error report -
 SQL Error: ORA-43907: This argument must be a literal or bind variable.

 

So, I am a little bit puzzled…we can use SYSDATE but not having a function around it…


Choosing another column value as the default

Now, how about choosing another column of the same table as the default value?

WITH data (str,dateval) AS (SELECT '15.01.2016', date '2015-07-15' FROM dual UNION ALL
                            SELECT '30.06.2016', date '2015-07-16' FROM dual UNION ALL
                            SELECT '30.02.2016', date '2015-07-17' FROM dual UNION ALL
                            SELECT 'gugus' , date '2015-07-18' FROM dual UNION ALL
                            SELECT '07.03.2016', date '2015-07-19' FROM dual UNION ALL
                            SELECT '15.10.2016', date '2015-07-20' FROM dual)
 SELECT str
      , dateval
      , TO_DATE(str DEFAULT dateval ON CONVERSION ERROR,'DD.MM.YYYY') AS converted
 FROM data
/
 
Error at Command Line : 9 Column : 29
Error report -
SQL Error: ORA-43907: This argument must be a literal or bind variable.

Ok, perhaps it needs to be a string using the same format mask?

WITH data (str,dateval) AS (SELECT '15.01.2016', date '2015-07-15' FROM dual UNION ALL
                            SELECT '30.06.2016', date '2015-07-16' FROM dual UNION ALL
                            SELECT '30.02.2016', date '2015-07-17' FROM dual UNION ALL
                            SELECT 'gugus' , date '2015-07-18' FROM dual UNION ALL
                            SELECT '07.03.2016', date '2015-07-19' FROM dual UNION ALL
                            SELECT '15.10.2016', date '2015-07-20' FROM dual)
 SELECT str
      , dateval
      , TO_DATE(str DEFAULT TO_CHAR(dateval,'DD.MM.YYYY') ON CONVERSION ERROR,'DD.MM.YYYY') AS converted
 FROM data
/
 
Error at Command Line : 9 Column : 29
Error report -
SQL Error: ORA-43907: This argument must be a literal or bind variable.

So, how can this be solved? 

Now, we could return NULL in case of a conversion error and then replace a null value by the value of the second column…like this:

WITH data (str,dateval) AS (SELECT '15.01.2016', date '2015-07-15' FROM dual UNION ALL
                            SELECT '30.06.2016', date '2015-07-16' FROM dual UNION ALL
                            SELECT '30.02.2016', date '2015-07-17' FROM dual UNION ALL
                            SELECT 'gugus'     , date '2015-07-18' FROM dual UNION ALL
                            SELECT '07.03.2016', date '2015-07-19' FROM dual UNION ALL
                            SELECT '15.10.2016', date '2015-07-20' FROM dual UNION ALL
                            SELECT NULL        , date '2015-07-21' FROM DUAL)
SELECT str
     , dateval
     , NVL(TO_DATE(str DEFAULT NULL ON CONVERSION ERROR,'DD.MM.YYYY'),dateval) AS converted
 FROM data
/

STR        DATEVAL    CONVERTED 
---------- ---------- ----------
15.01.2016 15.07.2015 15.01.2016
30.06.2016 16.07.2015 30.06.2016
30.02.2016 17.07.2015 17.07.2015
gugus      18.07.2015 18.07.2015
07.03.2016 19.07.2015 07.03.2016
15.10.2016 20.07.2015 15.10.2016
           21.07.2015 21.07.2015

7 rows selected. 

but…I only wanted to replace values that are not convertable by the second value, not null values…

WITH data (str,dateval) AS (SELECT '15.01.2016', date '2015-07-15' FROM dual UNION ALL
                            SELECT '30.06.2016', date '2015-07-16' FROM dual UNION ALL
                            SELECT '30.02.2016', date '2015-07-17' FROM dual UNION ALL
                            SELECT 'gugus' , date '2015-07-18' FROM dual UNION ALL
                            SELECT '07.03.2016', date '2015-07-19' FROM dual UNION ALL
                            SELECT '15.10.2016', date '2015-07-20' FROM dual UNION ALL
                            SELECT NULL , date '2015-07-21' FROM DUAL)
SELECT str
     , dateval
     , CASE 
          WHEN str IS NULL THEN NULL
          ELSE NVL(TO_DATE(str DEFAULT NULL ON CONVERSION ERROR,'DD.MM.YYYY'),dateval)
       END AS converted
 FROM data
/

A little bit complicated, but this way I get what I want.


Handling different date formats in one query

Now another test case, having an input stream using different date formats…

WITH data (str) AS (SELECT '15.03.2016' FROM dual UNION ALL
                    SELECT '2005/03/15' FROM dual UNION ALL
                    SELECT '15-MRZ-2016' FROM dual UNION ALL
                    SELECT '15-MAR-2016' FROM dual UNION ALL
                    SELECT '15. MARCH 2016' FROM dual UNION ALL
                    SELECT '15-MAALIS-2016' FROM DUAL UNION ALL
                    SELECT '15-BRE-2016' FROM DUAL)
SELECT str
     , COALESCE(TO_DATE(str DEFAULT NULL ON CONVERSION ERROR,'DD.MM.YYYY')
               ,TO_DATE(str DEFAULT NULL ON CONVERSION ERROR,'YYYY/MM/DD')
               ,TO_DATE(str DEFAULT NULL ON CONVERSION ERROR,'DD-MON-YYYY','NLS_DATE_LANGUAGE=ENGLISH')
               ,TO_DATE(str DEFAULT NULL ON CONVERSION ERROR,'DD-MON-YYYY','NLS_DATE_LANGUAGE=GERMAN')
               ,TO_DATE(str DEFAULT NULL ON CONVERSION ERROR,'DD-MON-YYYY','NLS_DATE_LANGUAGE=FINNISH')
       ) AS converted
 FROM data
/ 

STR            CONVERTED 
-------------- ----------
15.03.2016     15.03.2016
2005/03/15     15.03.2005
15-MRZ-2016    15.03.2016
15-MAR-2016    15.03.2016
15. MARCH 2016 15.03.2016
15-MAALIS-2016 15.03.2016
15-BRE-2016 

7 rows selected.  

 


Conclusion

The DEFAULT … ON CONVERSION ERROR is a very powerful feature, which will help us solving conversion errors very elegant. Of course we will still not be able to do conversions when the input value is not unambiguous (A string value of ‘03.05.2016’ could be the 3rd of May as well as the 5th of March) but still a very welcome feature.

 

ORACLE 12.2 – LISTAGG Enhancements

Tags

, , ,

As in earlier versions LISTAGG still returns a VARCHAR2 datatype with the known limitations (4 or 32 K). But instead of ending in an error on a size overflow we now have the chance to tell the function to truncate the return string in case it gets to long.

The syntax has changed slightly between the beta version of ORACLE 12.2 and the productive one, there some changes had to be done to this post.


LISTAGG default behaviour

The default behaviour is still the same resulting in an Oracle error on size overflow.

WITH test (id, str) as (SELECT MOD(rownum,7), RPAD('Text',10,'X') 
 FROM dual
 CONNECT BY ROWNUM < 10000)
SELECT id, LISTAGG(str,'-') WITHIN GROUP(ORDER BY 1) AS string_created
 FROM test 
 GROUP BY id
/ 

ORA-01489: result of string concatenation is too long

ON OVERFLOW TRUNCATE

The new option on overflow truncated leeds to a trunctated string having three dots at the end followed by the number of truncated characters in brackets.

The result lists only the end of the strings created.

WITH test (id, str) as (SELECT MOD(rownum,7), RPAD('Text',10,'X')
 FROM dual
 CONNECT BY ROWNUM < 10000)
SELECT id
     , LISTAGG(str,'-' ON OVERFLOW TRUNCATE) WITHIN GROUP(ORDER BY 1) AS string_created
 FROM test 
 GROUP BY id
/ 

ID STRING_CREATED
-- ------------------------------------------
 0 TextXXXXXX-TextXXXXXX-TextXXXXXX-...(1068)
 1 TextXXXXXX-TextXXXXXX-TextXXXXXX-...(1069)
 2 TextXXXXXX-TextXXXXXX-TextXXXXXX-...(1069)

Define overflow characters

We are free to define the text that should be shown as the overflow indicator. The text we want to see in case of an overflow is defined following the “ON OVERFLOW TRUNCATE” option.

WITH test (id, str) as (SELECT MOD(rownum,7), RPAD('Text',10,'X')
 FROM dual
 CONNECT BY ROWNUM < 10000)
SELECT id
     , LISTAGG(str,'-' ON OVERFLOW TRUNCATE ' [more] ') WITHIN GROUP(ORDER BY 1) AS string_created
 FROM test 
 GROUP BY id
/ 

ID STRING_CREATED
-- -----------------------------------------------
 0 TextXXXXXX-TextXXXXXX-TextXXXXXX- [more] (1068)
 1 TextXXXXXX-TextXXXXXX-TextXXXXXX- [more] (1069)
 2 TextXXXXXX-TextXXXXXX-TextXXXXXX- [more] (1069)

Omit the counting

The number of truncated characters can be omitted by adding WITHOUT COUNT (default is with count).

WITH test (id, str) as (SELECT MOD(rownum,7), RPAD('Text',10,'X')
 FROM dual
 CONNECT BY ROWNUM < 10000)
SELECT id
     , LISTAGG(str,'-' ON OVERFLOW TRUNCATE ' [more] ' WITHOUT COUNT) WITHIN GROUP(ORDER BY 1) AS string_created
 FROM test 
 GROUP BY id
/ 

ID STRING_CREATED
-- -----------------------------------------------
 0 TextXXXXXX-TextXXXXXX-TextXXXXXX- [more] 
 1 TextXXXXXX-TextXXXXXX-TextXXXXXX- [more] 
 2 TextXXXXXX-TextXXXXXX-TextXXXXXX- [more] 

Conclusion

Truncating the result of a LISTAGG function to avoid the overflow error is a pretty cool feature of Oracle 12.2 SQL. If you need to concatenate strings beyond the size limit of the VARCHAR2 datatype you will still need to write your own CLOB aggregate function but if it is enough to know, that there was more in the string before having concatenated it the on overflow truncate is really handy. 

Grouping and counting continuity

Tags

, , , , ,

A colleague of mine had the problem to group continuous entries of same type to be able to count number of continuos entries.

His Data:

TS        USERNAME RETURNCODE
--------- -------- ----------
05-NOV-15 SCOTT         28000
04-NOV-15 SCOTT         28000
03-NOV-15 SCOTT          1027 
02-NOV-15 SCOTT          1027
01-NOV-15 SCOTT          1027
31-OCT-15 SCOTT          1027
30-OCT-15 SCOTT             0
29-OCT-15 SCOTT          1027
28-OCT-15 SCOTT          1027
27-OCT-15 SCOTT             0
26-OCT-15 SCOTT             0
25-OCT-15 SCOTT          1027

Output needed:

TS        USERNAME RETURNCODE GROUP COUNT
--------  -------- ---------- ----- -----
05-NOV-15 SCOTT         28000     1     2
04-NOV-15 SCOTT         28000     1     2
03-NOV-15 SCOTT          1027     2     4
02-NOV-15 SCOTT          1027     2     4
01-NOV-15 SCOTT          1027     2     4
31-OCT-15 SCOTT          1027     2     4
30-OCT-15 SCOTT             0     3     1
29-OCT-15 SCOTT          1027     4     2
28-OCT-15 SCOTT          1027     4     2
27-OCT-15 SCOTT             0     5     2
26-OCT-15 SCOTT             0     5     2
25-OCT-15 SCOTT          1027     6     1

So basically the problem was to build groups on continuous identical returncodes and increase group number with every change. After those groups have been built the count column can be calculated using a COUNT(*) OVER (PARTITION BY the group-identification).


Solution 1 – ANALYTIC FUNCTION:

This solution compares the returncode value of the previous row (within same username ordered by timestamp) with the current value and returns “1” in case of a difference if the current value is unchanged NULL is returned. In the second step a running summary over this 1/NULL result is built.

WITH data (ts, username, returncode)        
       AS ( SELECT DATE '2015-11-05', 'SCOTT', 28000 FROM dual UNION ALL
            SELECT DATE '2015-11-04', 'SCOTT', 28000 FROM dual UNION ALL
            SELECT DATE '2015-11-03', 'SCOTT', 1027  FROM dual UNION ALL
            SELECT DATE '2015-11-02', 'SCOTT', 1027  FROM dual UNION ALL
            SELECT DATE '2015-11-01', 'SCOTT', 1027  FROM dual UNION ALL 
            SELECT DATE '2015-10-31', 'SCOTT', 1027  FROM dual UNION ALL
            SELECT DATE '2015-10-30', 'SCOTT', 0     FROM dual UNION ALL
            SELECT DATE '2015-10-29', 'SCOTT', 1027  FROM dual UNION ALL
            SELECT DATE '2015-10-28', 'SCOTT', 1027  FROM dual UNION ALL
            SELECT DATE '2015-10-27', 'SCOTT', 0     FROM dual UNION ALL
            SELECT DATE '2015-10-26', 'SCOTT', 0     FROM dual UNION ALL
            SELECT DATE '2015-10-25', 'SCOTT', 1027  FROM dual) 
SELECT ts
      ,username
      ,returncode
      ,SUM(start_entry) OVER (PARTITION BY username ORDER BY ts DESC) AS grp
  FROM (SELECT ts
             , username
             , returncode
             , CASE
                  WHEN returncode = LAG(returncode) OVER (PARTITION BY username
                                                              ORDER BY ts DESC)
                     THEN 0                   
                  ELSE 1
               END AS start_entry 
          FROM data) 
/

TS        USERN RETURNCODE GRP
--------- ----- ---------- ----------
05-NOV-15 SCOTT      28000          1
04-NOV-15 SCOTT      28000          1
03-NOV-15 SCOTT       1027          2
02-NOV-15 SCOTT       1027          2
01-NOV-15 SCOTT       1027          2
31-OCT-15 SCOTT       1027          2
30-OCT-15 SCOTT          0          3
29-OCT-15 SCOTT       1027          4
28-OCT-15 SCOTT       1027          4
27-OCT-15 SCOTT          0          5
26-OCT-15 SCOTT          0          5
25-OCT-15 SCOTT       1027          6          

Solution 2 – MODEL clause:

The model clauses has the possibility to compare the values of different rows and generate a value for a different column based on this comparision. In our case I compare the value of the returncode of the previous row to the value of the returncode of the current row. In case of identical values i reuse the value of the grp column of the previous row to set the grp value of the current row, in case of a difference I increase the previous rows value by 1.

WITH DATA (ts, username, returncode)
        AS( SELECT DATE '2015-11-05', 'SCOTT', 28000 FROM dual UNION ALL
            SELECT DATE '2015-11-04', 'SCOTT', 28000 FROM dual UNION ALL 
            SELECT DATE '2015-11-03', 'SCOTT', 1027  FROM dual UNION ALL
            SELECT DATE '2015-11-02', 'SCOTT', 1027  FROM dual UNION ALL
            SELECT DATE '2015-11-01', 'SCOTT', 1027  FROM dual UNION ALL
            SELECT DATE '2015-10-31', 'SCOTT', 1027  FROM dual UNION ALL
            SELECT DATE '2015-10-30', 'SCOTT', 0     FROM dual UNION ALL
            SELECT DATE '2015-10-29', 'SCOTT', 1027  FROM dual UNION ALL
            SELECT DATE '2015-10-28', 'SCOTT', 1027  FROM dual UNION ALL
            SELECT DATE '2015-10-27', 'SCOTT', 0     FROM dual UNION ALL
            SELECT DATE '2015-10-26', 'SCOTT', 0     FROM dual UNION ALL
            SELECT DATE '2015-10-25', 'SCOTT', 1027  FROM dual)
SELECT ts, username, returncode, grp
  FROM data
      MODEL PARTITION BY (username)
            DIMENSION BY (ROW_NUMBER() OVER (PARTITION BY username 
                                                  ORDER BY ts) AS rn)
            MEASURES (1 as grp,returncode,ts)
            RULES AUTOMATIC ORDER
            (grp[rn > 1]  = CASE WHEN returncode[CV()] = returncode[cv()-1] 
                                 THEN grp[CV()-1]
                                 ELSE grp[CV()-1] + 1
                            END)
/

Solution 3 “MATCH_RECOGNIZE (12c)”:

With the MATCH_RECOGNIZE clause we can do a count of the group members simultaneously to the group assignement.

WITH DATA (ts, username, returncode)
        AS(SELECT DATE '2015-11-05', 'SCOTT', 28000 FROM dual UNION ALL
           SELECT DATE '2015-11-04', 'SCOTT', 28000 FROM dual UNION ALL 
           SELECT DATE '2015-11-03', 'SCOTT', 1027  FROM dual UNION ALL
           SELECT DATE '2015-11-02', 'SCOTT', 1027  FROM dual UNION ALL
           SELECT DATE '2015-11-01', 'SCOTT', 1027  FROM dual UNION ALL
           SELECT DATE '2015-10-31', 'SCOTT', 1027  FROM dual UNION ALL
           SELECT DATE '2015-10-30', 'SCOTT', 0     FROM dual UNION ALL
           SELECT DATE '2015-10-29', 'SCOTT', 1027  FROM dual UNION ALL
           SELECT DATE '2015-10-28', 'SCOTT', 1027  FROM dual UNION ALL
           SELECT DATE '2015-10-27', 'SCOTT', 0     FROM dual UNION ALL
           SELECT DATE '2015-10-26', 'SCOTT', 0     FROM dual UNION ALL
           SELECT DATE '2015-10-25', 'SCOTT', 1027  FROM dual)
SELECT ts, username, returncode, grp, continuous_rows
  FROM DATA
  MATCH_RECOGNIZE (PARTITION BY username
                   ORDER BY ts DESC                    
                   MEASURES FINAL COUNT(*) AS continuous_rows
                           ,MATCH_NUMBER() AS grp
                   ALL ROWS PER MATCH
                   AFTER MATCH SKIP PAST LAST ROW
                   PATTERN (strt cont*)
                   DEFINE cont AS cont.returncode = PREV(cont.returncode)) x
/

Gap Detection – Tabibitosan Method

Tags

, ,

Another question from the ORACLE SQL & PL/SQL Forum.

Problem:

The original poster (OP) needed to find the first “missing” entry in a series of entries.

Entries are like:

  • Set 1: 0001, 0004, 0006  First missing entry: 0002
  • Set 2: 0001, 0002, 0006  First missing entry: 0003
  • Set 3: 0001, 0002, 0003   First missing entry: 0004
  • Set 4: 0002, 0003, 0004  First missing entry: 0001

Test Data:

WITH data (id,entry) AS (SELECT 1, '0001' FROM dual UNION ALL
                         SELECT 1, '0004' FROM dual UNION ALL
                         SELECT 1, '0006' FROM dual UNION ALL
                         SELECT 2, '0001' FROM dual UNION ALL
                         SELECT 2, '0002' FROM dual UNION ALL
                         SELECT 2, '0006' FROM dual UNION ALL
                         SELECT 3, '0001' FROM dual UNION ALL
                         SELECT 3, '0002' FROM dual UNION ALL
                         SELECT 3, '0003' FROM dual UNION ALL
                         SELECT 4, '0002' FROM dual UNION ALL
                         SELECT 4, '0003' FROM dual UNION ALL
                         SELECT 4, '0004' FROM dual)
SELECT id, entry
  FROM data
/

        ID ENTRY
---------- -----
         1 0001 
         1 0004 
         1 0006 
         2 0001 
         2 0002 
         2 0006 
         3 0001 
         3 0002 
         3 0003 
         4 0002 
         4 0003 
         4 0004 

 12 rows selected

Solution:

So first step would be to calculate the effective entry and the entry – “value an entry had if everything was gapless” (the column is called distance).

WITH data (id,entry) AS (SELECT 1, '0001' FROM dual UNION ALL
                         SELECT 1, '0004' FROM dual UNION ALL
                         SELECT 1, '0006' FROM dual UNION ALL
                         SELECT 2, '0001' FROM dual UNION ALL
                         SELECT 2, '0002' FROM dual UNION ALL
                         SELECT 2, '0006' FROM dual UNION ALL
                         SELECT 3, '0001' FROM dual UNION ALL
                         SELECT 3, '0002' FROM dual UNION ALL
                         SELECT 3, '0003' FROM dual UNION ALL
                         SELECT 4, '0002' FROM dual UNION ALL
                         SELECT 4, '0003' FROM dual UNION ALL
                         SELECT 4, '0004' FROM dual)
SELECT id, entry
     , TO_NUMBER(entry) - ROW_NUMBER() OVER (PARTITION BY id 
                                             ORDER BY entry) AS distance
  FROM data
/

        ID ENTRY   DISTANCE
---------- ----- ----------
         1 0001           0
         1 0004           2
         1 0006           3
         2 0001           0
         2 0002           0
         2 0006           3
         3 0001           0
         3 0002           0
         3 0003           0
         4 0002           1
         4 0003           1
         4 0004           1

12 rows selected

As long as the distance is 0 the entry belongs to a gapless series. To find the first missing entry, we are looking for the last entry with a distance of 0.

WITH data (id,entry) AS (SELECT 1, '0001' FROM dual UNION ALL
                         SELECT 1, '0004' FROM dual UNION ALL
                         SELECT 1, '0006' FROM dual UNION ALL
                         SELECT 2, '0001' FROM dual UNION ALL
                         SELECT 2, '0002' FROM dual UNION ALL
                         SELECT 2, '0006' FROM dual UNION ALL
                         SELECT 3, '0001' FROM dual UNION ALL
                         SELECT 3, '0002' FROM dual UNION ALL
                         SELECT 3, '0003' FROM dual UNION ALL
                         SELECT 4, '0002' FROM dual UNION ALL
                         SELECT 4, '0003' FROM dual UNION ALL
                         SELECT 4, '0004' FROM dual)
SELECT id, MAX(DECODE(distance,0,id,NULL)) AS last_in_serie
  FROM (SELECT id, entry
             , TO_NUMBER(entry) - ROW_NUMBER() OVER (PARTITION BY id
                                                         ORDER BY entry) AS distance
          FROM data)
  GROUP BY id
/

        ID LAST_IN_SERIE
---------- -------------
         1             1
         2             2
         3             3
         4     

By increasing this last_in_series value by 1 we get the first missing entry.

WITH data (id,entry) AS (SELECT 1, '0001' FROM dual UNION ALL
                         SELECT 1, '0004' FROM dual UNION ALL
                         SELECT 1, '0006' FROM dual UNION ALL
                         SELECT 2, '0001' FROM dual UNION ALL
                         SELECT 2, '0002' FROM dual UNION ALL
                         SELECT 2, '0006' FROM dual UNION ALL
                         SELECT 3, '0001' FROM dual UNION ALL
                         SELECT 3, '0002' FROM dual UNION ALL
                         SELECT 3, '0003' FROM dual UNION ALL
                         SELECT 4, '0002' FROM dual UNION ALL
                         SELECT 4, '0003' FROM dual UNION ALL
                         SELECT 4, '0004' FROM dual)
SELECT id, TO_CHAR(NVL(MAX(DECODE(distance,0,id,NULL)),0) + 1,'FM0000') as next_entry
  FROM (SELECT id, entry
             , TO_NUMBER(entry) - ROW_NUMBER() OVER (PARTITION BY id
                                                         ORDER BY entry) AS distance
          FROM data)
  GROUP BY id
/

        ID NEXT_ENTRY
---------- ----------
         1 0002      
         2 0003      
         3 0004      
         4 0001 

Partitioned Outer Join

Tags

, ,

This posts describes a solution to a problem posted on the ORACLE SQL and PL/SQL forums.


Problem:

The original poster (OP) has a table which has monthly data for agents.

WITH agt_dum(agent_id,mon,commission)
         AS (SELECT 1, 1, 50  FROM dual UNION ALL
             SELECT 1, 2, 100 FROM dual UNION ALL
             SELECT 1, 6, 25  FROM dual UNION ALL
             SELECT 2, 1, 10  FROM dual UNION ALL
             SELECT 2, 2, 20  FROM dual)
SELECT *
  FROM agt_dum
/

  AGENT_ID        MON COMMISSION
---------- ---------- ----------
         1          1         50
         1          2        100
         1          6         25
         2          1         10
         2          2         20

as you can see not every month has data for every AGENT_ID and the OP needed to have a query result like:

  AGENT_ID        MON COMMISSION
---------- ---------- ----------
         1          1         50
         1          2        100
         1          3          0
         1          4          0
         1          5          0
         1          6         25
         1          7          0
         1          8          0
         1          9          0
         1         10          0
         1         11          0
         1         12          0
         2          1         10
         2          2         20
         2          3          0
         2          4          0
         2          5          0
         2          6          0
         2          7          0
         2          8          0
         2          9          0
         2         10          0
         2         11          0
         2         12          0

meaning for each month an entry for each AGENT_ID and if no commission exists (no row exist) a 0 value for that specific column.


Solution(s):

The first thing that I though about was to have an outer join on the AGENT_ID. The driving table would be something that has all months (month numbers) from 1 to 12.

Outer Join with a “ALL_MONTHS” source:

WITH all_months (mon) 
             AS (SELECT rownum AS mon
                   FROM dual
                CONNECT BY rownum <= 12)
SELECT mon
  FROM all_months
/ 

MON
---------
        1
        2
        3
        4
        5
        6
        7
        8
        9
       10
       11
       12

This source could be used as the driving table in an outer join query

WITH agt_dum(agent_id,mon,commission)
         AS (SELECT 1, 1, 50  FROM dual union all
             SELECT 1, 2, 100 FROM dual union all
             SELECT 1, 6, 25  FROM dual union all
             SELECT 2, 1, 10  FROM dual union all
             SELECT 2, 2, 20  FROM dual)
    ,all_months (mon) AS (SELECT ROWNUM
                            FROM dual 
                          CONNECT BY ROWNUM <= 12)
SELECT ad.agent_id, am.mon, NVL(ad.commission,0) AS commission
  FROM            all_months am
  LEFT OUTER JOIN agt_dum    ad ON (ad.mon = am.mon)
 ORDER by ad.agent_id, am.mon
/

But a simple outer join will no solve the problem, as all the outer joined columns will be NULL…

  AGENT_ID        MON COMMISSION
---------- ---------- ----------
         1          1         50
         1          2        100
         1          6         25
         2          1         10
         2          2         20
                    3          0
                    4          0
                    5          0
                    7          0
                    8          0
                    9          0
                   10          0
                   11          0
                   12          0

As you can see you get an “empty” row for all months that are not available with any AGENT_ID … but what OP wanted was to have one row for every month for every AGENT_ID.


Partitioned Outer Join

So the solution for that is having a partitioned outer join like:

WITH agt_dum(agent_id,mon,commission)
         AS (SELECT 1, 1, 50  FROM dual union all
             SELECT 1, 2, 100 FROM dual union all
             SELECT 1, 6, 25  FROM dual union all
             SELECT 2, 1, 10  FROM dual union all
             SELECT 2, 2, 20  FROM dual)
    ,all_months (mon) AS (SELECT ROWNUM
                            FROM dual 
                          CONNECT BY ROWNUM <= 12)
SELECT ad.agent_id, am.mon, NVL(ad.commission,0) AS commission
  FROM            all_months am
  LEFT OUTER JOIN agt_dum    ad PARTITION BY (ad.agent_id) ON (ad.mon = am.mon)
 ORDER by ad.agent_id, am.mon
/

Using this query, ORACLE does an OUTER JOIN for every AGENT_ID in the AGT_DUM view and the result is exactly what OP wanted.

  AGENT_ID        MON COMMISSION
---------- ---------- ----------
         1          1         50
         1          2        100
         1          3          0
         1          4          0
         1          5          0
         1          6         25
         1          7          0
         1          8          0
         1          9          0
         1         10          0
         1         11          0
         1         12          0
         2          1         10
         2          2         20
         2          3          0
         2          4          0
         2          5          0
         2          6          0
         2          7          0
         2          8          0
         2          9          0
         2         10          0
         2         11          0
         2         12          0

Additional solutions:

Model Clause:

WITH agt_dum(agent_id,mon,commission)         
         AS (SELECT 1, 1, 50  FROM dual UNION ALL             
             SELECT 1, 2, 100 FROM dual UNION ALL             
             SELECT 1, 6, 25  FROM dual UNION ALL             
             SELECT 2, 1, 10  FROM dual UNION ALL             
             SELECT 2, 2, 20  FROM dual)
select agent_id  
     , mon  
     , commission  
 FROM agt_dum  
 MODEL PARTITION BY (agent_id)  
       DIMENSION BY (mon)  
       MEASURES (commission)   
       RULES ITERATE (12)  
       (  
           commission[ITERATION_NUMBER + 1] = NVL(commission[cv()], 0)  
       )  
 ORDER BY agent_id  
        , mon;

ORACLE 12c Release 1 – Session Level Sequences

Tags

, , ,

A sequence that starts at the starting point with every session that accesses the sequence…what is this good for?

Now, I could think at things like logging, where we could give every step of a session a ascending step_id, which would be stored in the log table along with the session_id to which the step belonged. Or we could have a transaction which creates a series of entries in the database and within the transaction those entries are numbered for whatever purpose. Generating Primary Keys for a Global Temporary Table would be another field where those kinds of sequences could be used.


Create a session level sequence

A session level sequence is generated using the new SESSION key word. This will restart the sequence for every session.

CREATE SEQUENCE session_seq
   START WITH   1
   INCREMENT BY 1
   SESSION
/ 

Usage

Example filling a global temporary table using a session level sequence as primary key generator:

CREATE SEQUENCE session_seq
 START WITH   1
 INCREMENT BY 1
 SESSION
/

Sequence created.

CREATE GLOBAL TEMPORARY TABLE employees_temp_table (
    employee_id  NUMBER DEFAULT session_seq.nextval NOT NULL
   ,first_name VARCHAR2(20)
   ,last_name VARCHAR2(25)
   )
   ON COMMIT PRESERVE ROWS
/

Table created.

REM ===========================================================================
REM Add rows to table
REM ===========================================================================
INSERT INTO employees_temp_table (first_name, last_name)
SELECT first_name, last_name
  FROM hr.employees
 WHERE rownum < 20
/

19 rows created.

REM ===========================================================================
REM Show the results
REM ===========================================================================
SELECT *
  FROM employees_temp_table
/

EMPLOYEE_ID FIRST_NAME           LAST_NAME
----------- -------------------- -------------------------
          1 Ellen                Abel
          2 Sundar               Ande
          3 Mozhe                Atkinson
          4 David                Austin
          5 Hermann              Baer
          6 Shelli               Baida
          7 Amit                 Banda
          8 Elizabeth            Bates
          9 Sarah                Bell
         10 David                Bernstein
         11 Laura                Bissot
         12 Harrison             Bloom
         13 Alexis               Bull
         14 Anthony              Cabrio
         15 Gerald               Cambrault
         16 Nanette              Cambrault
         17 John                 Chen
         18 Kelly                Chung
         19 Karen                Colmenares

19 rows selected.

If another session does the same thing, the employee_id will start at 1 again.

ORACLE 12c Release 1 – Truncate Cascade

Tags

, ,

Prior to ORACLE 12c truncating a table referenced by enabled foreign keys was no possible (even if the tables were empty).

ORA-02266: unique/primary keys in table referenced by enabled foreign keys

ORACLE 12c adds a new option to the TRUNCATE command

  • Parent and all children tables will be truncated if the foreign key between (all of) them is defined as ON DELETE CASCADE
  • Even if no parent exists (optional foreign key)
  • Even if the foreign key is defined to be novalidate
  • It is a truncate not a delete

Example

REM ===========================================================================
REM Create tables
REM ===========================================================================
CREATE TABLE dept_trunc_casc_demo (
   deptno NUMBER
  ,dname  VARCHAR2(30)
  ,CONSTRAINT dept_trunc_cast_demo_pk PRIMARY KEY (deptno)
  )
/


CREATE TABLE emp_trunc_casc_demo (
   empno  NUMBER
  ,ename  VARCHAR2(30)
  ,deptno NUMBER
  ,CONSTRAINT emp_trunc_casc_demo_pk PRIMARY KEY (empno)
  ,CONSTRAINT emp_dept_trunc_casc_fk FOREIGN KEY (deptno) REFERENCES dept_trunc_casc_demo
  )
/


REM ===========================================================================
REM Add data
REM ===========================================================================
INSERT INTO dept_trunc_casc_demo (deptno, dname) VALUES (10,'ACCOUNTING');
INSERT INTO dept_trunc_casc_demo (deptno, dname) VALUES (20,'SALES');

INSERT INTO emp_trunc_casc_demo (empno, ename, deptno) VALUES (1,'KUNZ',10);
INSERT INTO emp_trunc_casc_demo (empno, ename, deptno) VALUES (2,'MEIER',20);
INSERT INTO emp_trunc_casc_demo (empno, ename, deptno) VALUES (3,'HUBER',20);

COMMIT;

REM ===========================================================================
REM Trying to truncate master table
REM Will not work as with prior versions of ORACLE
REM ===========================================================================
TRUNCATE TABLE dept_trunc_casc_demo;

TRUNCATE TABLE dept_trunc_casc_demo
               *
ERROR at line 1:
ORA-02266: unique/primary keys in table referenced by enabled foreign keys

REM ===========================================================================
REM Trying to truncate master table with new keyword CASCADE
REM This will not work unless there is a FK with on delete cascade.
REM ===========================================================================
TRUNCATE TABLE dept_trunc_casc_demo CASCADE;

TRUNCATE TABLE dept_trunc_casc_demo CASCADE;
               *
ERROR at line 1:
ORA-14705: unique or primary keys referenced by enabled foreign keys in table "O12TEST"."EMP_TRUNC_CASC_DEMO"

REM ===========================================================================
REM Changing the foreign key to cascade delete
REM ===========================================================================
ALTER TABLE emp_trunc_casc_demo DROP CONSTRAINT emp_dept_trunc_casc_fk;

ALTER TABLE emp_trunc_casc_demo ADD CONSTRAINT emp_dept_trunc_casc_fk
   FOREIGN KEY (deptno) REFERENCES dept_trunc_casc_demo ON DELETE CASCADE;

REM ===========================================================================
REM Truncating Master Table with cascade keyword
REM ===========================================================================
TRUNCATE TABLE dept_trunc_casc_demo cascade;

REM ===========================================================================
REM Show number of rows in child table
REM ===========================================================================

SELECT COUNT(*)
  FROM emp_trunc_casc_demo;

COUNT(*)
--------
       0

Date arithmetic

Tags

, ,

The hard way and the smart way

A problem that often arises in the classroom is how to calculate the difference (hours, minutes and seconds) of two DATE values.


The hard way

At various places on the web you may find solutions like this:

WITH example_data AS (SELECT TO_DATE('01.01.2013 13:35:17','dd.mm.yyyy hh24:mi:ss') AS date_from
                            ,TO_DATE('01.01.2013 17:22:08','dd.mm.yyyy hh24:mi:ss') AS date_to
                        FROM DUAL)
SELECT TO_CHAR(TRUNC((date_to - date_from) * 24),'FM00') || ':' 
    || TO_CHAR(TRUNC(((date_to - date_from) * 24 - TRUNC((date_to - date_from) * 24)) * 60),'FM00') || ':' 
    || TO_CHAR(((date_to - date_from) * 24 * 60 - TRUNC((date_to - date_from) * 24 * 60)) * 60,'FM00') AS time
  FROM example_data;

TIME
-----------
03:46:51

Personally I would call this the hard way to solve the problem, because there is a much easier solution for this problem.


The smart way

WITH example_data AS (SELECT TO_DATE('01.01.2013 13:35:17','dd.mm.yyyy hh24:mi:ss') AS date_from
                            ,TO_DATE('01.01.2013 17:22:08','dd.mm.yyyy hh24:mi:ss') AS date_to
                        FROM DUAL)
SELECT TO_CHAR(TRUNC(SYSDATE) + (date_to - date_from),'HH24:MI:SS') as TIME
  FROM example_data;

TIME
--------
03:46:51

This solution makes use of the date arithmetic logic of ORACLE where the difference between two dates values is expressed as a number representing the number of days including a fraction of days. Furthermore adding a number to a date again results in a date data type. So if we add the difference of two date values to midnight and use TO_CHAR to display the time component of the result, this result will express the difference of those two dates. Now what should we do, if the difference is more than one day?
We just could make use of the day in year format after having added the difference between the two date values to the 31st of december.

WITH example_data AS (SELECT TO_DATE('01.01.2013 13:35:17','dd.mm.yyyy hh24:mi:ss') AS date_from
                            ,TO_DATE('01.01.2013 17:22:08','dd.mm.yyyy hh24:mi:ss') AS date_to
                        FROM DUAL
                      UNION ALL
                      SELECT TO_DATE('01.01.2013 13:35:17','dd.mm.yyyy hh24:mi:ss') AS date_from
                            ,TO_DATE('03.01.2013 17:22:08','dd.mm.yyyy hh24:mi:ss') AS date_to
                        FROM DUAL)
SELECT CASE
          WHEN TRUNC(date_to - date_from) > 0
             THEN TO_CHAR((TRUNC(SYSDATE,'year')-1) + (date_to - date_from),'DDD HH24:MI:SS')
          ELSE TO_CHAR(TRUNC(SYSDATE) + (date_to - date_from),'HH24:MI:SS')
       END as TIME
  FROM example_data;

TIME
------------
03:46:51
002 03:46:51

This solution represents in my opinion the smart way to calculate time difference between two date values.