[H-GEN] [NOVICE] Postgresql Historical data store

Gary Price (ICT) garybprice at bigpond.com
Thu Jun 13 23:09:56 EDT 2002


[ Humbug *General* list - semi-serious discussions about Humbug and     ]
[ Unix-related topics. Posts from non-subscribed addresses will vanish. ]


----- Original Message -----
From: <ben.carlyle at invensys.com>
To: <general at humbug.org.au>
Cc: <pgsql-novice at postgresql.org>
Sent: Wednesday, June 12, 2002 9:35 AM
Subject: [H-GEN] [NOVICE] Postgresql Historical data store


> [ Humbug *General* list - semi-serious discussions about Humbug and     ]
> [ Unix-related topics. Posts from non-subscribed addresses will vanish. ]
>
> G'day,
>
> As part of my work-related software development I've been asked to develop
> a system related to my company's product that is capable of storing
> time-ordered values for a set of inputs. The data is in the form of a
> IEEE-style double and must be timestamped with at least millisecond
> accuracy over the period of up to a year (or perhaps longer for other
> projects). I can expect around 20 million changes per day between all the
> inputs although many inputs will not change very much at all. I also need
> redundancy.
>

I saw this the first time it was posted.
Am I right in thinking there will be 20 million updates per day?
Have you considered storing the basic readings in disk files and populating
a database with derived data, say, computed at the end of each day? It would
make a big difference to the database provisioning.  Anything that need to
process the raw data would need to be high-speed and you will not get near
that with a pure database approach. I can't see that you will need flexible
indexing and queryig on the original data, so there would be no need to
incur all the overhead that requires.

Gary Price
ICT


--
* This is list (humbug) general handled by majordomo at lists.humbug.org.au .
* Postings to this list are only accepted from subscribed addresses of
* lists 'general' or 'general-post'.  See http://www.humbug.org.au/



More information about the General mailing list