Thursday, September 30, 2010

Getting ETL from redo log data

I'm designing an Inmon G.I.F like data warehouse and need to source data from a OLTP CRM like bespke app that I don't want to mess with. I've been investigating tools that can do changed data capture (CDC) from redo logs,no need to use Oracle's CDC. Using some kind of CDC reduces your data warehouse load sizes as you can just grab the deltas, Oracle RDBMS has built in CDC that comes in the Ent.Ed but may need DBMS streams and other configuration
Attunity CDC can be installed where the staging area is (no installation required on the machine where Oracle lives) Access to the redo log is through a SQL*Net connection.

Check out Attunity CDC and GoldenGate(now Oracle), superior to built in Oracle CDC. Still need to set the DB to add a supplemental log destination

ALTER DATABASE ADD SUPPLEMENTAL LOG DATA;

Attunity Pricing is based on the h/w config of the ETL platform where Attunity is installed; i.e. #cpu's and how many Oracle DB's get connected to from the ETL platform

Tuesday, September 14, 2010

bashreduce : mapreduce in a bash script

erikfrey's bashreduce at master - GitHub: "bashreduce : mapreduce in a bash script

- Sent using Google Toolbar"

POWER7 models and AIX (from IBM dW)

These are the minimums but higher service packs are recommended.
POWER7 Systems Power 750, 755 (Jan) Power 770,780 (March) Blade 700, 701,702 (June) Power 710, 720, 730, 740, 795 (Sept)
AIX 5.3 TL9 SP7 SP7 - -
AIX 5.3 TL10 SP4 SP4 SP5 SP5
AIX 5.3 TL11 SP2 SP2 SP5 SP5
AIX 5.3 TL12 * * * SP1
POWER7 Systems Power 750, 755 (Jan) Power 770,780 (March) Blade 700, 701,702 (June) Power 710, 720, 730, 740, 795 (Sept)
AIX 6.1 TL02 SP8 SP8 - -
AIX 6.1 TL03 SP5 SP5 SP7 SP7
AIX 6.1 TL04 SP2 SP3 SP7 SP7
AIX 6.1 TL05 * * * SP3
AIX 6.1 TL06 * * * *
POWER7 Systems Power 750, 755 (Jan) Power 770,780 (March) Blade 700, 701,702 (June) Power 710, 720, 730, 740, 795 (Sept)
AIX 7.1 TL00 * * * *
Notes:
  1. AIX 6 is highly recommended as the default install to ensure simple updates (you will then avoid the upgrade to 6.1, as AIX 5.3 is eventually phased out).
  2. - means this combination is not supported.
  3. * means no service pack needed ... but I always recommend adding the first service pack (as soon as it is available) to avoid early problems that are already fixed.
  4. AIX 5.2 TL10 SP8 is supported via AIX7, POWER7 and Versioned WPARs.
  5. Other AIX releases and Technology Levels are not supported.

alphaWorks : IBM Bluetooth Remote Invocation Tool : Overview

alphaWorks : IBM Bluetooth Remote Invocation Tool : Overview: "- Sent using Google Toolbar"

Five secret data storage and data management vendors

Five secret data storage and data management vendors: "- Sent using Google Toolbar"

Oracle Reintroduces Solaris Support for non-Oracle x86 Hardware (IDEAS Insights)

Oracle Reintroduces Solaris Support for non-Oracle x86 Hardware (IDEAS Insights): "- Sent using Google Toolbar"

Porting Perl To Python and the nose Unit Testing Framework.

Porting Perl To Python: "- Sent using Google Toolbar"

Open source IQ test | Open Source - InfoWorld

Open source IQ test | Open Source - InfoWorld: "Open source IQ test

- Sent using Google Toolbar"

Oracle Siebel 8 support on IBM AIX6

AIX 6 and Siebel version 8.0.0.10 will play together,AIX 6 and Siebel version 8.0.0.5 or below will not, AIX 5.3 TL12 may be the best option for support longevity.