Antipatterns SQL & PL/SQL – Redefine oracle defined exceptions

During my earlier work as a consultant and in my todays job as a technical supporter to the development team I encountered various SQL & PL/SQL antipatterns. I would like to share some of them with you.


The first time I hit this kind of code I was a little bit confused … it never came to my mind to do something like redefining named exceptions like NO_DATA_FOUND or TOO_MANY_ROWS or even OTHERS.

But after many years I became pretty hardened when it comes to what to expect during a code review.

My own NO_DATA_FOUND Exception

So guess what this peace of code does…

declare
   L_Variable number;
   no_data_found EXCEPTION;
begin
   select empno
     into l_variable
     from scott.emp
    where ename = 'TYRION LENNISTER';
exception
    when no_data_found
       then sys.dbms_output.put_line('Tyrion does not exist');
end;

It ends with raising a no_data_found exception upon the caller…

Error report -
ORA-01403: no data found
ORA-06512: at line 5
01403. 00000 -  "no data found"
*Cause:    No data was found from the objects.
*Action:   There was no data from the objects which may be due to end of fetch.

Hold a minute…why does it raise an exception? There is an exception handler that should take care of no_data_found …
well yes … but …
That is a user defined exception which is different from the named exception that was automatically raised by Oracle. It shares the same name but it is different.

Now how about this?

<<outer>>
begin
   <<inner>>
   declare
      L_Variable number;
      no_data_found EXCEPTION;
   begin
      select empno
        into l_variable
        from scott.emp
       where ename = 'TYRION LENNISTER';
    exception
       when no_data_found
          then sys.dbms_output.put_line('Tyrion does not exist');
    end inner;
exception
   when no_data_found
      then sys.dbms_output.put_line('Yes, Tyrion really does not exist');
end outer;

This time we get the message

Yes, Tyrion really does not exist

because in the outer block the no_data_found exception declared in the inner block is not known and therefore we fall back to Oracles named exception NO_DATA_FOUND which is handled in the outer block.


My own OTHERS

How about an user defined others exception?

<<outer>>
begin
   <<inner>>
   declare
      L_Variable number;
      others EXCEPTION;
   begin
      select empno
        into l_variable
        from scott.emp
       where ename = 'TYRION LENNISTER';
   exception
      when others 
         then sys.dbms_output.put_line('Tyrion does not exist');
      when no_data_found 
         then sys.dbms_output.put_line('Hello, what did you expect');
   end inner;
exception
   when others 
      then sys.dbms_output.put_line('Yes Tyrion really does not exist');
end outer;

I put the others handler ahead of the no_data_found just to make sure it is “my” others (you cannot do that with the named others) and …

Error report -
ORA-06550: line 14, column 7:
PLS-00370: OTHERS handler must be last among the exception handlers of a block
ORA-06550: line 3, column 1:
PL/SQL: Statement ignored
06550. 00000 -  "line %s, column %s:\n%s"
*Cause:    Usually a PL/SQL compilation error.
*Action:

So obviously that others was not mine but the one of oracle. Next try:

<<outer>>
begin
   <<inner>>
   declare
      L_Variable number;
      others EXCEPTION;
   begin
      select empno
        into l_variable
        from scott.emp
       where ename = 'TYRION LENNISTER';
   exception
      when inner.others 
         then sys.dbms_output.put_line('Tyrion does not exist');
      when no_data_found 
         then sys.dbms_output.put_line('Hello, what did you expect');
   end inner;
exception
   when others 
      then sys.dbms_output.put_line('Yes Tyrion really does not exist');
end outer;

This time I got no error

Hello, what did you expect

So I passed the others exception handler (my own) and fell into the no_data_found handler. The same happens when you write your code like this:

<<outer>>
begin
   <<inner>>
   declare
      L_Variable number;
      others EXCEPTION;
   begin
      select empno
        into l_variable
        from scott.emp
       where ename = 'TYRION LENNISTER';
   exception
      when inner.others 
         then sys.dbms_output.put_line('Tyrion does not exist');
      when others
         then sys.dbms_output.put_line('Hello, what did you expect');
   end inner;
exception
   when others 
      then sys.dbms_output.put_line('Yes Tyrion really does not exist');
end outer;

But, honestly …

This is really confusing and in my eyes it is absolutely needless to confuse our colleagues by redefining named exceptions. So get rid of that!


Conclusion

Knowing bad habits is the best way to avoid them.

Most of the examples I found are based on:

  • copy/paste coding
  • sticking to things that worked in the past (without questioning them)
  • ignoring new features

This (and the other posts on the anti-patterns) should be a help to get over them.

ORA12R2 – IMPdp may change segment_column_id

This is a case we faced just a couple of days ago and somehow I cannot believe what I see.

We have a table containing a invisible column which was added to the table and therefor was placed at the end of the segment.


Testcase

drop table myemp;

create table myemp (empno    number(4)     not null
                   ,ename    varchar2(10)
                   ,job      varchar2(10)
                   ,mgr      number(4)
                   ,hiredate date
                   ,sal      number(6)
                   ,comm     number(4)
                   ,deptno   number(2));

alter table myemp add constraint myemp_pk primary key (empno);

Insert into MYEMP (EMPNO,ENAME,JOB,MGR,HIREDATE,SAL,COMM,DEPTNO) 
values (7369,'SMITH','CLERK',7902,date '1980-12-17',800,null,20);
...
commit;

alter table myemp add birthday date invisible;

update myemp set birthday = add_months(hiredate,-18);

commit;

col column_name format a15

select column_name
      ,hidden_column
      ,column_id
      ,segment_column_id
      ,internal_column_id
  from user_tab_cols
 where table_name = 'MYEMP';

COLUMN_NAME HID COLUMN_ID SEGMENT_COLUMN_ID INTERNAL_COLUMN_ID
----------- --- --------- ----------------- ------------------
EMPNO       NO          1                 1                  1
ENAME       NO          2                 2                  2
JOB         NO          3                 3                  3
MGR         NO          4                 4                  4
HIREDATE    NO          5                 5                  5
SAL         NO          6                 6                  6
COMM        NO          7                 7                  7
DEPTNO      NO          8                 8                  8
BIRTHDAY    YES                           9                  9

9 rows selected.


expdp usr/pwd tables=MYEMP directory=EXP dumpfile=myemp.dmp logfile=myemp.log reuse_dumpfiles=yes

drop table myemp;

impdp usr/pwd tables=MYEMP directory=EXP dumpfile=myemp.dmp logfile=myemp.log

select column_name
      ,hidden_column
      ,column_id 
      ,segment_column_id 
      ,internal_column_id
  from user_tab_cols
 where table_name = 'MYEMP';

COLUMN_NAME HID COLUMN_ID SEGMENT_COLUMN_ID INTERNAL_COLUMN_ID
----------- --- --------- ----------------- ------------------
BIRTHDAY    YES                           1                  1
EMPNO       NO          1                 2                  2
ENAME       NO          2                 3                  3
JOB         NO          3                 4                  4
MGR         NO          4                 5                  5
HIREDATE    NO          5                 6                  6
SAL         NO          6                 7                  7
COMM        NO          7                 8                  8
DEPTNO      NO          8                 9                  9

9 rows selected.

Notes to this behaviour:

  • Oracle 19 does it correct, segment_column_id, internal_column_id remains unchanged
  • It is a problem of impdp
    • I tried to import an Oracle 12 export using the import utility of Oracle 19 and it turned out ok
    • I tried to import an Oracle 19 export using the import utility of Oracle 12 and it turned out wrong
  • The data is correct
  • I have not tried with Oracle 18c

Is it a bug?

Yes it is in my opinion – I just have to convince Oracle Support of this. They were able to reproduce but offered a workaround … which is not really an option for us. We will move to Oracle 19c in the next couple of months and we have this problem in our development environment only but still I was really surprised facing something like this…

Antipatterns SQL & PL/SQL – “SELECT *” even if you only need a few columns

During my earlier work as a consultant and in my todays job as a technical supporter to the development team I encountered various SQL & PL/SQL antipatterns. I would like to share some of them with you.


Another sign of programmers laziness is the wide usage of “select *”. This antipattern can be found in inline views, database views, cursors …

The only column needed in the following example is the employee_id, so why do we read the whole row?

This is a waste for many reasons:

  • expensive columns like CLOB/BLOB/XMLTYPE
  • chained rows in the table
  • maybe we could benefit of an index-only access if we pared the query down to the minimum
  • storage that is needed to hold the row

Which Columns are really needed?

For this PL/SQL block we would have to return employee_id only from the implicit cursor.

begin
   for EmpRec in (select *
                    from employees
                   where salary < l_minSalary) loop
      raise_salary(in_employee_id => EmpRec.employee_id);
   end loop;
end;

In the following example the only relevant columns needed as result of the inline view on GR_BRCT_STWE (which has 39 columns) are GR_LNR_1 to join GR_STAMM and GR_LNR_2 to filter against a bind variable (which could be done in the inline view anyway) .

select count(*)
  from (select *
          from GR_BRCT_STWE
         where USERBK_NR = :2) verb
  join GR_STAMM mfh on (    verb.GR_LNR_1 = mfh.GR_LNR
                        and mfh.GR_TYP_CD = 3 
                        and mfh.USERBK_NR = :2)
  join GR_GBDE mfh_gbde on (    mfh_gbde.GR_LNR = mfh.GR_LNR
                            and mfh_gbde.USERBK_NR = :2)
 where verb.GR_LNR_2 = :1
  and mfh_gbde.WHG_ANZ is not null;

If we have to use most but not all of the columns of a table it might be cumbersome to write down all the columns, but nowadays we have great support by IDEs like SQL Developer.

SQL Developer may help you…

If you write down a select * from table you will discover a yellow wavy line underneath the asterisk. Hovering your mouse pointer over this line will enable you to apply the complete column list, which may be arranged afterwards.


Efficient coding? Sorry, I call it laziness.

Yes it may be more work to specify exactly which columns you need, but not doing so is not a sign of efficiency but only of laziness.


Conclusion

Knowing bad habits is the best way to avoid them.

Most of the examples I found are based on:

  • copy/paste coding
  • sticking to things that worked in the past (without questioning them)
  • ignoring new features

This (and the other posts on the anti-patterns) should be a help to get over them.

Antipatterns SQL & PL/SQL – Substitute NULL with empty string

During my earlier work as a consultant and in my todays job as a technical supporter to the development team I encountered various SQL & PL/SQL antipatterns. I would like to share some of them with you.


One of the worst things that may be seen often in PL/SQL code is using an empty string as a substitution of a NULL value. There are places where it technically works and therefore is simply ugly and others where it is just wrong.


Ugly NULL substitutions

The ugly usages are those where an empty string is used as an assignment to a variable or a parameter. Even if this is not technically wrong it may be a reason why inexperienced colleagues may end substituting NULL by empty string not only in assignements but also when it comes to conditions.

So ugly NULL substitutions look like this:

...
   L_Variable := '';
   F_GetTableValues('P1' ,'P2' ,'' ,P3 ,'P4');
...
   
select decode(a.Column,'',0,a.OtherColumn)
   from table a;

...
   procedure OpenForm (P_UserId      in varchar2
                      ,P_StartUpArgs in varchar2 default ''
                      ,P_Commit      in number default 0);
...

Wrong NULL substitutions

Substituting NULL by an empty string within a condition is not only ugly, it is just wrong and it will not lead to the expected results. The reason therefore is, that this kind of conditions will not work with the IS / IS NOT keyword (as needed for NULL comparison) but with an equal/unequal operator.

Obviously is is not really unknown, that comparing with an empty is string is kind of problematic, this might be the reason why we can often find things like:

if L_Variable = '' or L_Variable is null then
...
end if;

if L_Variable <> '' or L_Variable is not null then
...
end if;

An example using the HR sample locations table

select city, country_id, state_province
  from locations
 where state_province = '';

no rows selected


select city, country_id, state_province
  from locations
 where state_province is null;

CITY       COUNTRY_ID  STATE_PROVINCE
---------  ----------  --------------
Roma       IT
Venice     IT
Hiroshima  JP
Beijing    CN
Singapore  SG
London     UK


select city, country_id, state_province
  from locations
 where state_province <> '';

no rows selected

Why substitute?

Maybe two single quotes are shorter than the keyword NULL (2 vs 4 letters) but as soon as we think about adding an additional or clause we should really get rid of the NULL substitutions.

And this please not only at the places where the substitutions is wrong but also where it is only ugly…


Conclusion

Knowing bad habits is the best way to avoid them.

Most of the examples I found are based on:

  • copy/paste coding
  • sticking to things that worked in the past (without questioning them)
  • ignoring new features

This (and the other posts on the anti-patterns) should be a help to get over them.

Antipatterns SQL & PL/SQL – Reusing table aliases in the same SQL statement

During my earlier work as a consultant and in my todays job as a technical supporter to the development team I encountered various SQL & PL/SQL antipatterns. I would like to share some of them with you.


Be creative…at least a little bit.

Another antipattern that I see over and over again, is the reuse of table aliases within the same statement … yes syntactically everything is fine, but it is a nightmare to read a query like this – besindes the fact, that table aliases like a,b,c are awful.

select e.first_name, e.last_name
  from employees e
 where e.job_id in (select a.job_id
                      from jobs a
                     where a.min_salary between 5000 and 7500
                        or a.max_salary between 7500 and 9000)
   and e.department_id in (select a.department_id
                             from departments a
                             join locations b on (b.location_id = a.location_id)
                            where b.country_id = 'UK')
   and e.manager_id in (select a.employee_id
                          from employees a
                         where a.hire_date < date '2005-01-01');

To me a table alias is not only a method to clarify the source of a column upon the SQL engine but also a way to ease reading the statement. Therefore the benefit to the reader is much higher than the higher effort it takes for the writer using longer aliases which are more meaningful.

select emp.first_name, emp.last_name
  from employees emp
 where emp.job_id in (select job.job_id
                        from jobs job
                       where job.min_salary between 5000 and 7500
                          or job.max_salary between 7500 and 9000)
   and emp.department_id in (select dpt.department_id
                               from departments dpt
                               join locations loc on (loc.location_id = dpt.location_id)
                              where loc.country_id = 'UK')
   and emp.manager_id in (select mgr.employee_id
                            from employees mgr
                           where mgr.hire_date < date '2005-01-01');

Conclusion

Knowing bad habits is the best way to avoid them.

Most of the examples I found are based on:

  • copy/paste coding
  • sticking to things that worked in the past (without questioning them)
  • ignoring new features

This (and the other posts on the anti-patterns) should be a help to get over them.

Antipattern SQL & PL/SQL – Using the LIKE operator even if searching for equality.

During my earlier work as a consultant and in my todays job as a technical supporter to the development team I encountered various SQL & PL/SQL antipatterns. I would like to share some of them with you.


I guess that queries like the following may be found throughout every application.

select *
  from emp
 where ename like 'SCOTT';

select table_name, comments
  from dictionary
 where table_name like 'ALL_TABLES';

Those queries are confusing, error-prone and sometimes even wrong.


Confusing

If we find a LIKE expression in a query, the expectations would be to see some wildcards in the condition. Not having any wildcards in the query leads to a certain suspicion.

Error-Prone

When using the like operator instead of an equal sign we have to make sure that every wildcard character that should not be used as a wildcard is escaped. The most common wild card is the percent sign (%) … the one that is often missed is the underscore.

So basically this query is not only searching for an object called “ALL_TABLES” but also for something starting with “ALL” having any charcater at position 4 and ending with “TABLES”.

select table_name, comments
  from dictionary
 where table_name like 'ALL_TABLES';

To correct this query we need to escape the underscore

select table_name, comments
  from dictionary
 where table_name like 'ALL\_TABLES' escape '\';

or even better use an equal instead of the like operator.


Same same but different

Do comparisons without wildcard always return the same result regardless of using the LIKE operator or the equal sign?

Well….no, they dont.
There are not many cases where the CHAR datatype is still used, but nevertheless it is important to know, how this datatype behaves when it comes to equal comparisons.

Blank-Padded comparison semantics (Oracle – SQL Language Reference)

With blank-padded semantics, if the two values have different lengths, then Oracle first adds blanks to the end of the shorter one so their lengths are equal. Oracle then compares the values character by character up to the first character that differs. The value with the greater character in the first differing position is considered greater. If two values have no differing characters, then they are considered equal. This rule means that two values are equal if they differ only in the number of trailing blanks. Oracle uses blank-padded comparison semantics only when both values in the comparison are either expressions of data type CHAR, NCHAR, text literals, or values returned by the USER function.

Nonpadded comparison semantics (Oracle – SQL Language Reference)

With nonpadded semantics, Oracle compares two values character by character up to the first character that differs. The value with the greater character in that position is considered greater. If two values of different length are identical up to the end of the shorter one, then the longer value is considered greater. If two values of equal length have no differing characters, then the values are considered equal. Oracle uses nonpadded comparison semantics whenever one or both values in the comparison have the data type VARCHAR2 or NVARCHAR2.

Example for blank-padded comparison

As you can see it does not matter how many spaces we add to the search string Hello as the database type is a CHAR and a blank-padded comparison is used.

create table test_char (x char(10));

insert into test_char (x) values ('Hello');

select *
from test_char
where x = 'Hello';

X
----------------
Hello

select *
from test_char
where x = 'Hello ';

X
----------------
Hello

However using LIKE for the comparison changes everything. A like comparison will never use the blank-padding semantic and therefore we are not able to find a row.

select *
from test_char
where x like 'Hello';

no rows selected

What is the datatype of a string literal?

According to the documentation (ORACLE SQL Language Reference) a string (or text or character) literal has the CHAR datatype and therefore also is compared using the blank-padding semantic.


Conclusion

Knowing bad habits is the best way to avoid them.

Most of the examples I found are based on:

  • copy/paste coding
  • sticking to things that worked in the past (without questioning them)
  • ignoring new features

This (and the other posts on the anti-patterns) should be a help to get over them.

Antipatterns SQL & PL/SQL – Existance Check

During my earlier work as a consultant and in my todays job as a technical supporter to the development team I encountered various SQL & PL/SQL antipatterns. I would like to share some of them with you.

The first antipattern is the “check existance by doing a complete count” pattern.


How do we check whether a particular value exists in a table?

E.g. we would like to know whether there is a person in UK or DE which earns at least 80% of the maximum salary defined by the job. If we found that kind of rows, we would have to call a given procedure.

An anti-pattern that can be often found to solve this kind of question, is reading way to many rows to take your decision.

...
   select count(*)
     into L_Counter
     from EMPLOYEES   e
     join JOBS        j on (j.JOB_ID        = e.JOB_ID)
     join DEPARTMENTS d on (d.DEPARTMENT_ID = e.DEPARTMENT_ID)
     join LOCATIONS   l on (l.LOCATION_ID   = d.LOCATION_ID)
    where e.SALARY > (j.MAX_SALARY * .8)
      and l.COUNTRY_ID in ('DE','UK');

   if L_Counter > 0 then
      procedure1();
   end if;
...

To reduce the number of rows to be read, we should add a rownum filter to stop searching as soon as we found a hit.

...
   select count(*)
     into L_Counter
     from EMPLOYEES   e
     join JOBS        j on (j.JOB_ID        = e.JOB_ID)
     join DEPARTMENTS d on (d.DEPARTMENT_ID = e.DEPARTMENT_ID)
     join LOCATIONS   l on (l.LOCATION_ID   = d.LOCATION_ID)
    where e.SALARY > (j.MAX_SALARY * .8)
      and l.COUNTRY_ID in ('DE','UK')
      and rownum = 1;

if L_Counter > 0 then
   procedure1();
end if;
...

A different approach would be to use a subquery to answer the question connecting it to dual using an EXIST clause. One of the main differences to the count/rownum approach is, that this kind of query may raise a no_data_found exception and we therefore do not have to check a counter variable. We can rely on a “normal continuation” if a row was found. If no row exists an exception will be raised.

...
   begin
      select 1
        into L_Counter
        from dual
       where exists (select 'Employee with > 80% salary in UK/DE found'
                       from EMPLOYEES   e
                       join JOBS        j on (j.JOB_ID        = e.JOB_ID)
                       join DEPARTMENTS d on (d.DEPARTMENT_ID = e.DEPARTMENT_ID)
                       join LOCATIONS   l on (l.LOCATION_ID   = d.LOCATION_ID)
                      where e.SALARY > (j.MAX_SALARY * .8)
                        and l.COUNTRY_ID in ('DE','UK'))

      procedure1();
   exception
      when no_data_found then
         ...
   end;
...


To update or not to update?

Nowadays the question whether to insert or update a row can easily be solved using the merge command. However I came accross situations where the insert had to be done using an API whilst the update was done directly.

Situations like that are another place where you can find the “check existance by count”-antipattern.

...
   select count(*)
     into L_Counter
     from EMPLOYEES   e
    where e.EMPLOYEE_ID = in_emprec.employee_id;

   if L_Counter > 0 then
      update EMPLOYEES e
         set ...
       where e.EMPLOYEE_ID = in_emprec.employee_id;
   else
      InsertEmployee(in_emprec => in_emprec);
   end if;
...

This solution has two problems:

  1. In case of an update we need to process the employees table twice (first for the count, second for the update)
  2. We ignore the fact, that between the count and the follow-up action the situation could change due to data manipulations done by other sessions.

As a solution for the insert or update question (if no merge is possible) my suggestion would be to try the update and do the API call if the update command ended in updating 0 rows.

...
   update EMPLOYEES e
      set ...
    where e.EMPLOYEE_ID = in_emprec.employee_id;
   
   if SQL%ROWCOUNT = 0 then
      InsertEmployee(in_emprec => in_emprec);
   end if;
...

Or (even better) thinking about handling update/insert in the API using a merge command.


Testing nested tables for existence of an element

A way of testing existance of elements I see quite often is the “SQL way”.  The collection is scanned with a SQL doing a count(*) (without stop key). Maybe this has been an evolutionary way a couple of years ago, but today …

...
   select count(*)
     into L_Count
     from table(L_NestedTable)
    where column_value = L_Value;
 
    if (L_COUNT > 0) then
...

Today the easiest way is to use MEMBER OF to check for existens of an certain element in a nested table.

...
   if L_value member of L_NestedTable then
...

Conclusion

Knowing bad habits is the best way to avoid them.

Most of the examples I found are based on:

  • copy/paste coding
  • sticking to things that worked in the past (without questioning them)
  • ignoring new features

This (and the other posts on the anti-patterns) should be a help to get over them.

ORACLE 18c – TO_UTC_TIMESTAMP_TZ

Tags

,

TO_UTC_TIMESTAMP_TZ takes a ISO8601 formatted date string and converts it to a TIMESTAMP WITH TIMEZONE datatype.

According to documentation this is the ebnf of to_utc_timestamp_tc

to_utc_timestamp_tz

The varchar parameter may be either a date (only) or a date with time combination. The format of the parameter is:

Date only:

YYYY-MM-DD

Date / time:

YYYY-MM-DDThh:mm:ss[.s[s[s[s[s[s]]]]]][Z|(+|-)hh:mm]

Example

with data (d) as (select '2020-02-28T21:32:52'       from dual union all
                  select '2020-02-28T21:32:52+02:00' from dual union all
                  select '2020-02-28T19:32:52Z'      from dual union all
                  select '2020-02-28T19:32:52+00:00' from dual union all
                  select '2020-02-28T21:32:52.12679' from dual union all
                  select '2020-02-28')
select to_utc_timestamp_tz(d) as converted
  from data;

CONVERTED 
---------------------------------------------
28.02.2020 21:32:52.000000000 GMT
28.02.2020 21:32:52.000000000 +02:00
28.02.2020 19:32:52.000000000 GMT
28.02.2020 19:32:52.000000000 GMT
28.02.2020 21:32:52.126790000 GMT
28.02.2020 00:00:00.000000000 GMT

6 rows selected.

DEFAULT RETURN VALUE ON CONVERSION ERROR

As most of the other conversion functions documentation says that TO_UTC_TIMESTAMP_TZ supports the on conversion error clause too. But when trying to use it you will get an error.

with data (d) as (select '2018-10-26T21:32:52'       from dual union all
                  select '2018-10-26T21:32:52+02:00' from dual union all
                  select '2018-10-26T19:32:52Z'      from dual union all
                  select '2018-10-26T19:32:52+00:00' from dual union all
                  select '2018-10-26T21:32:52.12679' from dual)
select to_utc_timestamp_tz(d default null on conversion error) converted
  from data;

ORA-00907: missing right parenthesis
00907. 00000 - "missing right parenthesis"
*Cause: 
*Action:
Error at Line: 11 Column: 30

Documentation Bug

I have filed an SR with Oracle Support regarding this error. It has been classified as documentation bug:

DOC 30936692 – TAKE OUT “[ DEFAULT RETURN_VALUE ON CONVERSION ERROR ]” OPTIONAL CLAUSE FROM TO_UTC_TIMESTAMP_TZ

and will be removed from future (>20) documentations.


Conclusion

Somehow I hopped, that Oracle would add the missing functionality concerning the conversion error handling to the TO_UTC_TIMESTAMP_TZ function rather than declaring it as a documentation bug. 

Not having a decent way to handle exceptions (besides building a user defined function with an exception handler) will keep me away from using this function.

ORACLE 19c – LISTAGG Enhancement

With ORACLE 19c another gap in the listagg feature was filled. Now we are able to do a native distinct operation on the values in the list, so that the list no longer contains duplicates.

In prior releases we had to deduplicate the values before using them in the listagg, which was rather dreadful when doing other group functions in the same query.


LISTAGG Deduplication

Imagine we would have to list all product subcategories of the SH.Products table group by the product category.

select PROD_CATEGORY
     , listagg(PROD_SUBCATEGORY,' | ') as SubCategories
  from SH.PRODUCTS
 group by PROD_CATEGORY;

PROD_CATEGORY                 SUBCATEGORIES
----------------------------  -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Photo                         Camcorders | Camera Batteries | Camera Batteries | Camera Batteries | Camera Batteries | Camera Batteries | Camera Media | Camera Media | Camera Media | Cameras
Electronics                   Game Consoles | Home Audio | Home Audio | Y Box Accessories | Y Box Accessories | Y Box Games | Y Box Games | Y Box Games | Y Box Games | Y Box Games | Y Box Games | Y Box Games | Y Box Games                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           
Software/Other                Accessories | Accessories | Accessories | Accessories | Accessories | Bulk Pack Diskettes | Bulk Pack Diskettes | Documentation | Documentation | Documentation | Documentation | Documentation | Documentation | Operating Systems | Recordable CDs | Recordable CDs | Recordable CDs | Recordable CDs | Recordable CDs | Recordable CDs | Recordable CDs | Recordable DVD Discs | Recordable DVD Discs | Recordable DVD Discs | Recordable DVD Discs | Recordable DVD Discs                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             
Peripherals and Accessories   Accessories | Accessories | Accessories | Accessories | Accessories | CD-ROM | CD-ROM | CD-ROM | CD-ROM | CD-ROM | CD-ROM | Memory | Memory | Modems/Fax | Modems/Fax | Monitors | Monitors | Printer Supplies | Printer Supplies | Printer Supplies | Printer Supplies                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   
Hardware                      Desktop PCs | Portable PCs

As we can see, there are duplicates in the subcategory lists. So we had to do a deduplication in the past.

select PROD_CATEGORY
     , listagg(PROD_SUBCATEGORY,' | ') as SubCategories
  from (select distinct PROD_CATEGORY, PROD_SUBCATEGORY
          from SH.PRODUCTS)
 group by PROD_CATEGORY;

PROD_CATEGORY                SUBCATEGORIES
---------------------------  -------------------------------------------------------------------------------------------------------------
Electronics                  Y Box Accessories | Game Consoles | Home Audio | Y Box Games
Software/Other               Bulk Pack Diskettes | Recordable CDs | Recordable DVD Discs | Accessories | Documentation | Operating Systems
Photo                        Camera Batteries | Camera Media | Camcorders | Cameras
Peripherals and Accessories  CD-ROM | Accessories | Modems/Fax | Monitors | Printer Supplies | Memory
Hardware                     Portable PCs | Desktop PCs

Now, imagine we would like to add the sum of the list_prices to this query … to get the data consistent, we had to do the sum on the list_prices in two steps. First per distinct prod_category/prod_subcategory to get a deduplicated list of subcategories and then on the category level using the pre-calculated sums.

select PROD_CATEGORY
     , listagg(PROD_SUBCATEGORY,' | ') as SubCategories
     , sum(SumProdListPrice)           as SumProdListPrice
  from (select PROD_CATEGORY
             , PROD_SUBCATEGORY
             , sum(PROD_LIST_PRICE) as SumProdListPrice
          from SH.PRODUCTS
         group by PROD_CATEGORY
                , PROD_SUBCATEGORY) p1
 group by PROD_CATEGORY;

Oracle 19c spelling

Now this got much easier with Oracle 19c, the only thing we have to do is adding distinct to the listagg function

select PROD_CATEGORY
     , listagg(distinct PROD_SUBCATEGORY,' | ') as SubCategories
     , sum(PROD_LIST_PRICE)                     as SumProdListPrice
  from SH.PRODUCTS
 group by PROD_CATEGORY;

If you need to have the list ordered, then you have to use the within group syntax.

select PROD_CATEGORY 
     , listagg(distinct PROD_SUBCATEGORY,' | ') within group (order by PROD_SUBCATEGORY) as SubCategories  
     , sum(PROD_LIST_PRICE) as SumProdListPrice
  from SH.PRODUCTS
 group by PROD_CATEGORY;

WITHIN GROUP clause

Starting with version 19c the WITHIN GROUP clause is optional and only needed if the list has to be ordered.


Conclusion

Adding the possibility to remove duplicates from the list using the distinct keyword looks like a tinyness. But as most of the time duplicates in the list are not wanted this tinyness is really useful.

Best way to loop over a dense nested table

Tags

,

“What is the best way to process a dense nested table?” seems to be a simple question with an quite simple answer…

Looping First to Last

One way to implement this kind of loop is making use of the collection methods first and last to write code like:

begin
   for i in myArray.first() .. myArray.last() loop
      ... do whatever you need to do
   end loop;
end;

This works as long as the array is initialized and not empty.

If the array is not initialized you get a

ORA-06531: Reference to uninitialized collection

exception. This because the first/last methods can only be used on an initialized collection. If the collection is initilized but empty, the exception thrown is:

ORA-06502: PL/SQL: numeric or value error

the reason for this error is, that on an empty array first and last return null as the result and this is not feasible for a numeric for loop.

So the correct implemetation using the first/last version of a loop would be:

begin
   if myArray is not null
      and myArray is not empty
   then
      for i in myArray.first() .. myArray.last() loop
         ... do whatever you need to do
      end loop;
   end if;
end;

but…we can do better (and shorter).


Looping 1 to count

A slightly different approach is to use 1 as the lower and count as the upper border. If count is called on an empty collection we get 0 as result which has the effect that the loop body is never executed. But as with last/first we get an error when count is used on a collection that has not been initialized. 

So the solution using 1..count would look like this:

begin
   if myArray is not null
   then
      for i 1 .. myArray.count() loop
         ... do whatever you need to do
      end loop;
   end if;
end;

but again…we can do better (and shorter).


Looping using cardinality

Instead of using count to get the number of entries of a nested table we can also use the cardinality function. The advantage of cardinality is that this function can be called on not initialized collections too. The result of cardinality in this case would be null. Null however is not a valid value for a loop border so the cardinality call has to be surrounded by a nvl/coalesce function.

begin
   for i 1 .. nvl(cardinality(myArray),0) loop
      ... do whatever you need to do
   end loop;
end;

This is the shortes way to loop over a collection and it is a version that handles not initialized and empty collections too.