C#/.Net flat file DB and msSQL or Postgres convert

May 26th, 2006
Posted in .NET, Linux, Windows

Currently I have a custom built flat file fixed-field database. It simple saves usernames and passwords, and a few other pieces of info.
Currently a c# application does random access read and writes on the
file database, and this application has been running for many months
problem free.
It needs to be enhanced. There are 2 real main things that need to
be done. 1) A in ram data structure to allow chaching, so that
reads are done WITHOUT disk access, and archiving, and 2) the data has to sync with either a postgres DB or a MSsql DB. Perferably a Postgres DB. The sync, need only to in one direction at this time.

(1) on start up of the application, all fields should be read into a data structure, probably a .Net "hash" table data structure with a
login name as the search key, and the data held in each structure will
be simply username, password, and a few other field.
When the program goes to READ from the existing application, it goes to hard-drive read each time, we need this to infact only go to the
ram hash data structure for a read (thus not involving a hard drive access). Now if there is some data change, then the data has to be
updated in the ram data structure (essentially acting as a cache)
as well as writting to the file DB on the hard drive.
Also it is required that a archive of the DB be created and saved on
disk with a time stamp, every (a) one hour, (b) after 10 transaction
that involve changing data - which ever is LONGER. So if 40 peices of data change in one hour, that will still only result in ONE creation
of a archive file at that next hourly interval. IF one piece of data
changes, again at the next hour interval. In otherwords, if data changes, don’t write anymore frequently then once per hour. IF no data changes, then no need to write an archive.
The archive just needs to be a copy of the existing flat file DB,
but with a compression done on it, i.e. just provide a means to call
out to a system command to run a "gzip", "bzip", "zip" on the file.
The archive, after written (but before compressed), should be read
back into the application, to validate its integrity, this
integrity check should not effect the main "live" database,
or in RAM status of the database, but rather simple a isolated,
read in of the DB, to assure its integrity, and if it does not
reload and prove to be accurate data, then the archive must be recreated, and on two failures in a row, a flag can be set to
indicate a system error. If the validation is sucessful, then
the system call is made to compress the data, i.e.
gzip db_dump_YYYY_MM_DD-HH_MM_SS, this compress is of course
taken care of by a simple OS call to that program, so the application
just has to issue a call out to the OS.
On startup of the application, if the primary file DB is corrupt,
i.e. application crashed in middle of a write, then the application
should then go back to the last archive, uncompress it, by simply
issuing a system command i.e. "gunzip db_dump_YYYY_MM_DD-HH_MM_SS"
the last archive written, and unzip it, and load it in.
There should also be an option to reload from any given archive.
In particular, a routine is written to make a list of all the
archives, and load there names in. And a routine, that as a
paramter, given a archive name, load in that DB and also
refreshes the ram cache data structure, essentially allow to
revert to an older archive set of data. In addition, once the
number of archive files is greater then 20, the application issues
a system call out to run a script. The script will not be part
of what needs to be programmed, it will just be a simple perl script
that will move all files older then latest 20 to another directory,
again the programmer doesnt have to do this, but just know
to make a OS call out to a script name once number of archive files
is more then say 20.

(2) Sync. File DB to a postgres or MS SQL server.
If a request is made to the local DB for a username that IS NOT
in it, we need to make a DB call out to postgres or MsSQL and
check to see if that record exists there. If it does, it takes the
data and also updates the local file DB and the ram cache.
If it doesn’t, then nothing happens.
Also every 1/2 hour (or similar), the application has to
poll the postgres or MS-SQL DB and do a select based on a time stamp,
that essential says : any records inserted or changed in last 1/2 hour
that data needs to be copied to the local file DB. So a simple select
of all recently changed data comes from the Postres or MSSQL DB,
and sits in a DataSet in the application, and then each
row of that data set has to be insert/update the local record,
update if the local record already exists, else create new.

A simple text log file shuold also be written too that
creates a audit record for each thing done by (1) and (2) above,
except "READS" from local DB don’t need to be logged,
just "writes" to local DB, and and new data that gets taken from the
Postgres or MS-SQL DB.

This project requires straight forward c# abilities and experience
with data base calls from .Net/c# to postgres or MS-SQL,
however the code has to be able to run on c# on linux, which
should happen if its coded not using any MS specific .Net DB access
classes. I can provide all necessary Postgres examples, etc,
but if a programmer only has MS-SQL ability, we will have to
verify that the MS-SQL.
http://www.mono-project.com/ADO.NET
perhaps ODBC can be attempted, if it is straight forward to
get ODBC to postgres and MS-SQL.

This is probably 3 solid days of work for someone who has done similar, perhaps 4 days. Probably is 8-10 pages of code, maybe less.
Very simple testing is required, I will supply detailed testing,
so the quote on this job doesn’t have to include a lot of
testing, as other things are happening on this project, so testing
will happen as part of another effort. The code just generally has
to work, and be reasonably coded, and commented.
Keep in mind all existing working code is provided that
works with the flat file, and plenty of postgres DB examples
can also be provided.
So with that and experience with .Net and simple sql DB
calls on either/or postgres, MS-SQL, everything is straight forward.
Even more exact specification will be made available as well.

Need this started in approx. 1 week. or sooner and completed over
1-2 weeks after that. however part (2) could be done a bit later
if needed. I can be a bit flexable and add some more dollars if
issue pop up, but generally it should be very straightforward.
If someone has used MONO/c# and odbc’d to postgres AND MS-SQL
this also may be valuable and add to $$$, but at a min. we just
require DB connectivity to Postgres, or MS-SQL. The programmer
must have their own access to these DB’s on their own systems,
however if really necessary, a postgres access can be provided.
It would be usually expected that someone experienced in this stuff
would have a personal install of a postgres DB (i.e. its free).

(Budget: $100-300, Keywords: .NET, Linux, Windows)







Related Projects

postgres to mysql conversion


Hi,I have an old product that was created for me for a postgres database that I need converted into mysql format. It's mainly the create table etc conversion that is needed to begin with but we might even need a conversion of the php files as well (templates etc). But bid for the database design conversion to begin with. (Budget: $20-100, Keywords: PHP)

JSF Application Development With Database Backend


Need to create a web-application using JSF, from scratch. You will be provided with html-designed pages that you can use to create the application. So there will not be much UI/web-page designing involved. This is more of an application development project. The application will have atleast 30 or more screens (probably more, as we enhance it). Development work will also involve creating a Postgres database, and connecting to it from the JSF application. We will provide you with database tables in MS-Access. You can use that to easily create the needed tables in Postgres.Experience with Sun Creator 2 (for

Looking for .net and Postgres SQL expert


I am looking for a .net and PostgeSQL expert who can help me bring to life my software I have installed on my machine! Additional information submitted: (Budget: $30-100, Keywords: .NET) Place Your Bid

Freeside install and configure Phase1and 2


50% of bid paid to Install freeside open source ISP Billing Script with Mason and all the needed perl modules for RT ticketing via ssh access.Configure apache,intergrate with exsisting freeradius (mysql, but can change to postgres)server and install online signup page on seperate www server.Install Iptable forward rule to use freeside private server and bring freeside server to a fully working, usable state to complete Phase 1Completed Phase 1 and verified usable state payment = 35% of bid price paid.Phase 2 = install payment moduleCompleted Phase 2 and verified usable state = 15% of bid price paidSystem pref:Will accept a

User Administration


This project requires, free radius, php, mysql, postgres, mssql and encrypting the php scripts.

access application


We are in need of an access application adp application that interacts with a ms sql database back end.1. The access application should be in adp and should allow the user to browse for a local .mdb file and then run dts to automatically move all tables over to the SQL server. If the tables do exist in the sql server they should be overwritten. Note that the SQL database will also house some persistent tables (that do not have corresponding table in mdb) that need to be kept intact through this process.2. I want a

Access ADP application / MS SQL


NOTE: THIS APPLICATION NEEDS TO BE WRITTEN IN VBA IN ACCESS!We are in need of an access adp application that interacts with a ms sql database back end. I'd like to keep the project under $300 and would like very rapid turnaround.1. The access application should be in adp and should allow the user to browse for a local .mdb file, present the user with options that allow for selection of tables (there should be a select all button) and then run dts to automatically move all tables over to the SQL server. If the selected tables do exist in

access adp application with MS SQL


We are in need of an access adp application that interacts with a ms sql database back end. I'd like to keep the project under $300 and would like very rapid turnaround.1. The access application should be in adp and should allow the user to browse for a local .mdb file, present the user with options that allow for selection of tables (there should be a select all button) and then run dts to automatically move all tables over to the SQL server. If the selected tables do exist in the sql server they should be overwritten. Note that

Mapped Network Drive Namespace Application


We need to develop a windows explorer namespace extension application which mirrors a network share on a remote server. Our central server exposes a share for uploading and downloading of files from any network location. The application we need will create a folder or mapped drive attached to the Windows Explorer root, which displays the contents of the mapped drive so the user can drag and drop or copy and paste files from their local computer into the mapped drive. The application needs to not only provide access to the network share via the namespace extension, but we also need

Passenger’s Query System


This project basically deal with the flat file which is to be prepared by the Indian Airlines Employes. Through this project we scan that flat file using java file handling and after scanning we made an intermediate flat file. After the completion of intermediate file we again scan that file and put the data to the database and after that we put the query on to the database. here s1.txt is the source flat file and outp.txt is the intermediate flat file and ind is the database file. The whole...Place Your Bid Post similar project 

Compression Application


I'd like a compression application that can read and write as many archive formats as possible. Aftr installation the application will make its self the default application for all supported archive file types and begin a 7, 15, or 30 day trial. After the trial is over the application will cease to function and prompt user to register.Place Your Bid Post similar project 

Back Up Database File


I' ve got a script in the server which is saving dataentered through a form in a flat file Pipe delimiteddatabase.Example:Name|Last Name|Address|TelephoneThe problem is that the server has crashed twise andall the data was lost in both occasions.I am looking for a way that the script will send thesame data entered in the form not only to the flatfile database in the server, but to a Back Up file inmy PC. If the server crashes again I will not loosethe data because I have a local copy. (Budget: $20-100, Keywords: Perl/CGI)

Data Collection Application (Parsing)


I need a stand alone application or CGI that can take a URL (same site)then collect and organize the information in fields that can be exported as a csv file.The ideal application should be able to receive a URL (so I can call it from my own database) Then go to the web site, collect the data and automatically save the results on a local machine. (Budget: $300-1500, Keywords: C/C++, Data Processing, Perl/CGI, Visual Basic)

7z Like Application by legacy96


Requirements: Create Application to: -read and write as many archive formats (7z, zip, rar, etc) aspossible. -After installation the application will make it self the default application for all supported archive file types and begin a XX day trial... (Budget: $250-750, Jobs: C/C++, Visual Basic, Windows)Place Your Bid Post similar project 

PHP Rate Search flat file


I need a real simple search. The flat file used as the database is tab deliminated ( we can easily change the formatting ). In the flat file is an organization name and a percent number. I need a search box I can put on any page and then just a results page. There will no hyperlink or anything, you will just enter the name of your organization and search, it will return any relevant organization names along with the percentage. I need to be able to edit the results page to match the site ( I will do this

Virtual Drive / Explorer Namespace App


We need to develop a windows explorer namespace extension application which mirrors a network share on a remote server. Our central server exposes a share for uploading and downloading of files from any network location. The application we need will create a folder or mapped drive attached to the Windows Explorer root, which displays the contents of the mapped drive so the user can drag and drop or copy and paste files from their local computer into the mapped drive. The application needs to not only provide access to the network share via the namespace extension, but we also need

Windows Shell Namespace Client / Server app


We need to develop a windows explorer namespace extension application which mirrors a network share on a remote server. Our central server exposes a share for uploading and downloading of files from any network location. The application we need will create a folder or mapped drive attached to the Windows Explorer root, which displays the contents of the mapped drive so the user can drag and drop or copy and paste files from their local computer into the mapped drive. The application needs to not only provide access to the network share via the namespace extension, but we also need

Website Recovery from Archive.org


We need a script to recover a website from the archive.org waybackmachine.The script should take the website url and a target within the local filesystem and get as many files from archive.org as possible.All javascript, urlrewritung, htmlcomments and so on from archive.org should be removed. (Budget: $30-100, Keywords: Data Processing, Linux, PHP)

Asterisk Call Shop


Looking for an Asterisk Based Call Shop application, To allow a call shop operator to view in realtime calls in progress from individual booths, enable/disable booths, generate an invoice after the completion of the call, enter their own destination purchase and sell rates, reporting to include Call performance atempts/completions per dial code (country/biling region defined) and the average length of call.interface to be via WWW on a local lan, with remote access for multiple site owners.I am looking either for new development or purchase of existing product.

xslt transformation xml to flat file


XSLT Transfomation from XML file to a flat file. This is an custom invoice format xml to a flat file format.Will provide XML spec and lots of samples. Flat file spec and have only a few ff samples. I can also provide other sample xslt files that go from this same XML invoice format to EDI 4010 or 3030 is requested. Also have pretty good detailed pdf specs on the format of the flat file we are going into.Deliverable is the xslt file.

  • PHP Web Development and Hosting Freelance career guide
  • Advertise Here

  • Dedicated Django developer and ruby programmer available for hire for $5/hour.


FreeLance Home Jobs is Digg proof thanks to caching by WP Super Cache!