sql - ETL - Views or persist tables? -


when building data warehouse see 2 main approaches etl-process:

1. view - view of views - view of views of views - ...

approach 1 in database , has advantage don't have redundant data, lead performance issues.

2. stage table (copy of data) - clear table (copy of data) - dwh table (copy of data) - ...

approach 2 done many tools stored procedures , jobs or etl-tool ssis. advantage here it's easy understand process can visualise pretty good. have overall etl-performance , many predefined tasks etc. problem example, change of process more complex persistent tables have changed.

in real world see mix of both, when many people have worked on process. of course depends on situation (size of tables, how similar processes designed in company, how complex etl-process, ...).

i prefer copy tables, keep etl-process simple , if possible in etl-tool (usually ssis in case) designed purpose.

but best practice , why?

views views of views not scale volume of data in dwh. when comes of dwh mean talking huge volumes of data. integration of data multiple sources common usecase dwh. stage->tranform-->fact/dim 1 of common way how dwh built store data. yes change when talk hdfs , other technologies, views not able give desired performance in dwh. have seen many systems , of them have multi step etl process first data dwh sources , clean/process/conform/transform data via etl/others in dimensional/other model.


Comments

Post a Comment

Popular posts from this blog

node.js - Using Node without global install -

i'm working on few node & electron (atom shell) projects , i'm curious how them work without user having install node globally. i've seen few applications including node directory in project contains node.exe binary. how go setting node project use binary? edit: thought should add, i've searched everywhere answer this, unrelated results.
Read more

How to access a php class file from PHPFox framework into javascript code written in simple HTML file? -

i've website developed using phpfox v3.0.7 . now i'm trying implement 'sse(server sent events)' 1 of php files. reference i'm giving below necessary code part of file. file titled process.class.php , present @ location /var/www/module/notification/include/service/process.class.php present on server having ip http://34.124.40.142/ . <?php /** * [phpfox_header] */ header('content-type: text/event-stream'); header('cache-control: no-cache'); defined('phpfox') or exit('no dice!'); /** * * * @copyright [phpfox_copyright] * @author raymond benc * @package phpfox_service * @version $id: service.class.php 67 2009-01-20 11:32:45z raymond_benc $ */ class notification_service_process extends phpfox_service { /** * class constructor */ public function __construct() { $this->_stable = phpfox::gett('notification'); } public function add($...
Read more

java - Null response to php query in android, even though php works properly -

Image
table i'm trying retrieve above data using 2 php files below. want pass email parameter , return bets user = email. work fine when tested advanced rest client, problem not php. however, when try through android 2 java classes below, null pointer exception (logcat below) , json object returned null. i'm not sure error is, think i'm not passing parameter php , that's why it's producing null response. logcat 05-31 15:10:50.227 26074-26074/com.example.albert.betterapp e/androidruntime﹕ fatal exception: main process: com.example.albert.betterapp, pid: 26074 java.lang.runtimeexception: unable start activity componentinfo{com.example.albert.betterapp/com.example.albert.betterapp.displayallbets}: java.lang.nullpointerexception @ android.app.activitythread.performlaunchactivity(activitythread.java:2292) @ android.app.activitythread.handlelaunchactivity(activitythread.java:2350) @ android.app.activitythread.acce...
Read more