Styleforum › Forums › General › General Chat › Database & Datawarehouses
New Posts  All Forums:Forum Nav:

Database & Datawarehouses

post #1 of 14
5/25/10 at 8:37pm
Thread Starter 
A quick question for all the IT gurus out there. I am looking to start a datawarehouse. At present, I will have approximately 30 tables a week with approximate dimensions of 3600 x 7. I do not have a lot of previous databasing experience--could anyone help me:

  1. How difficult will it be to create such a database?
  2. What would be the optimal databasing software to utilize under the above-stated requirements?
  3. How expensive would such software run?

Thanks for any help that can be provided.
Reply
Reply
post #2 of 14
5/25/10 at 9:55pm
I'm curious why you will have 30 new tables each week.
Reply
Reply
post #3 of 14
5/25/10 at 9:58pm
The other key pieces of info would be access types / speeds / frequencies, and also what size are your variables.
Reply
Reply
post #4 of 14
5/25/10 at 11:09pm
Thread Starter 
Perhaps I am using incorrect terminology--let me explain the situation a little better.

I am attempting to track a portfolio of approximately 30 products and am attempting to capture the data at store level (approximately 3600 stores). I will have new data each week. Most variables will be either dummy variables, or pretty simple 2-4 digit variables.

What exactly do you mean by access types, speeds, and frequencies? By speed, I am presuming you mean the speed to process a query? If don't want it to be a snail's pace, by any stretch, but it doesn't have to be lightning quick. Regarding frequency, if I understand you correctly--queries would be ran multiple times daily (perhaps 20).

Thanks again.
Reply
Reply
post #5 of 14
5/25/10 at 11:29pm
I'm not an IT guy at all, but I deal a lot with scientific data stored in databases.

Access type - reading and writing. Some databases you primarily just read the data over and over again. Other databases you are going to constantly hit for storage of new information. Some are a blend. Frequency does refer to how often you are hitting it with queries. This is more hardware side.

As for size, to put it in perspective, I think MS Excel can handle 256 columns by 64000 rows, using decent sized variables (floats or doubles probably).

Your requirements don't sound too bad at all. Access would probably handle it, unless you want something web based, then maybe MySQL or MS SQL Server would be a good idea.

I'd avoid adding new tables all the time.

MySQL - free
MS Access - comes with Office
MS SQL Server - no idea, I use MySQL if I need big databases.
Reply
Reply
post #6 of 14
5/25/10 at 11:41pm
Quote:
Originally Posted by fairholme_wannabe View Post
A quick question for all the IT gurus out there. I am looking to start a datawarehouse. At present, I will have approximately 30 tables a week with approximate dimensions of 3600 x 7. I do not have a lot of previous databasing experience--could anyone help me:

  1. How difficult will it be to create such a database?
  2. What would be the optimal databasing software to utilize under the above-stated requirements?
  3. How expensive would such software run?

Thanks for any help that can be provided.

1. Those are aren't requirements.
2. Datawarehouse doesn't mean what you seem to think it means.
3. Dimensions wrt to databases also doesn't mean what you seem to think it means.
4. Software is going to be the least of your costs.
Reply
Reply
post #7 of 14
5/25/10 at 11:56pm
Couldn't you just have 30 tables total, 1 for each product, and add a row for every store each week? Assuming that you aren't hitting this constantly for data from thousands of possible sites, we're talking a simple Access database right? Then you can just run queries that match dates and stores or whatever data you need.
Reply
Reply
post #8 of 14
5/26/10 at 12:06am
Too complicated for SF.

OP, I suggest you read a book on access or something. What you want is basic but i don't think anyone is going to step you through it here. Any decent book on access will cover basic database design issues in the first couple of chapters.

If multiple people will be using the database at once, you'll need something like mysql or ms sql, but it sounds like it's just for your use otherwise you'd have a programmer to do this stuff for you.
Reply
Reply
post #9 of 14
5/26/10 at 12:12am
Quote:
Originally Posted by videocrew View Post
Couldn't you just have 30 tables total, 1 for each product, and add a row for every store each week? Assuming that you aren't hitting this constantly for data from thousands of possible sites, we're talking a simple Access database right? Then you can just run queries that match dates and stores or whatever data you need.
Why would you do that? Have one products table and they all get a productId. Then link to other tables as needed. You could have another table that would contain a record for each product and store each week. I'm not sure I follow the requirements exactly but something like this is where he should be heading tables: Table1:Products ProductId,Description,etc Table2:Stores StoreId,address,phone etc Table3:ProductsSoldByWhatever ProductId, StoreId, NumberOfProductSold,Date/week/importIdentifier This way you can add products or store and it won't fuck up your design. You can see how many products sold by a certain store over a period of time etc. you might even group the stores into geographic regions since there are so many of them. These are all design issues that will depend on how the data is to be used. That's why the OP should read a book and think about it himself a bit.
Reply
Reply
post #10 of 14
5/26/10 at 1:38am
Quote:
Originally Posted by GQgeek View Post
I'm not sure I follow the requirements exactly but something like this is where he should be heading
...

+1 to this. I'm not in IT, but I deal with fairly large scale data on an daily basis. OP, google something on database normalization or even read about MySQL a bit before claiming strange things like you'll be adding 20 tables a week.

The only scenario in which it could potentially make sense to have multiple tables for multiple products would be if you were working with something high-frequency and in real-time, like equities tick data.
Reply
Reply
post #11 of 14
5/26/10 at 9:37am
Thread Starter 
As I said--my terminology may be (and indeed was, very!) wrong. I appreciate the input, all. Glad to know the solution is fairly simple. Once again--appreciate the advice.
Reply
Reply
post #12 of 14
5/30/10 at 7:50am
Get a real life person to help because you are over your head. You are unaware of your own ignorance in this area.

Quote:
Originally Posted by fairholme_wannabe View Post
As I said--my terminology may be (and indeed was, very!) wrong. I appreciate the input, all. Glad to know the solution is fairly simple. Once again--appreciate the advice.

Table definitions are just one part of the solution. However I have a feeling that you don't have the correct table definitions because you never gave a proper set of requirements. Your comment: "I am attempting to track a portfolio of approximately 30 products and am attempting to capture the data at store level (approximately 3600 stores)." is for example is very ambiguous. What do you mean by "attempting to capture the data". What is the data? Is it only who holds what? Is it who holds what when? Does it include transactions, aggregates of transactions or nothing? Does it include current or future inventory levels?
Where does the data comes from? How is that data going to come from there? How is the data to be used? How is this to be implemented, maintained and backed up?
Reply
Reply
post #13 of 14
5/30/10 at 10:37am
It sounds like you have 3-4 tables with a relatively small amount of data. Use Access if you have it or look at MySQL because it's free.
Reply
Reply
post #14 of 14
5/30/10 at 11:25am
Reply
Reply
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: General Chat
Styleforum › Forums › General › General Chat › Database & Datawarehouses