| |
|
Syteview DataGrabbers are small programs that you use to automate the replication of data from your Progress database
to your Microsoft SQL Server. Each DataGrabber handles the replication of one table from your Progress database
to your MS-SQL Server database.
DataGrabbers always move an entire table from the Progress database server to the Syteview database in the SQL Server.
This replication process can take anywhere from several seconds to an hour or more depending on the number of records in the
table that is being replicated and the processing power of the computers running the Progress database server and the Microsoft
SQL Server.
Note: For times when you need to keep a table in your Progress database synchronized with a table in the Syteview
database in real-time Syteview provides you with DataLink programs.
In order for a DataGrabber to retrieve data from your Progress database the database server must be running.
The Progress database server must also have been installed with the SQL-92 Server Engine. When a DataGrabber
runs the first thing it does is drop (delete) the previous copy of the table in the SQL Server database. This
assures you that the new copy of the table is an exact copy of the current table in your Progress database. In
addition to creating an exact copy of the table, the DataGrabbers will create indexes and grant permissions on the
table created.
You define the indexes that each DataGrabber will create on its particular table with a text file named Indexes.txt.
This file must exist in the same sub-directory as the DataGrabber. The format of the Indexes.txt files is quite
simple. See the Indexes.txt file for one of the pre-existing DataGrabbers in the \DataGrabbers sub-directory in
the Syteview installation directory on your SQL Server for details. Include an empty file named Indexes.txt in
the Data Grabber’s sub-directory if you don’t want any indexes to be created for the table.
You define the permissions that each DataGrabber will grant on the table it is creating with a text file named
Permissions.txt. This file must exist in the same sub-directory as the DataGrabber. The format
of the Permissions.txt files is also quite simple. See the Permissions.txt file for one of the pre-existing
DataGrabbers in the \DataGrabbers sub-directory in the Syteview installation directory on your SQL Server for details.
Include an empty file named Permissions.txt in the DataGrabber’s sub-directory if you don’t want any permissions to be
granted on the table.
Note: There is a batch file (Get.SQL.Tables.bat) in the \DataGrabbers directory of the Syteview installation
directory that when run will startup all of the DataGrabber programs. There is also an individual batch file in
each DataGrabber’s home directory that can be used to run the DataGrabbers one at a time. These batch files will
work unchanged if you installed the Syteview SQL components to the D: drive, \Syteview directory. If you installed
Syteview to a different drive or directory you will need to edit these programs to reflect your situation. By
adding these batch files to the Windows Task Scheduler you can easily automate the replication of any of your Progress tables
to the Syteview database in your SQL Server.
Syteview includes Data Grabbers for many of the tables in your Progress Syteline database. The following table
lists these Data Grabbers and their location in the Syteview installation directory on your SQL Server:
| DataGrabber |
Table Replicated |
Location |
| GetAPTRXPs |
APTRXP |
\APTRXPS |
| GetARINVs |
ARINV |
\ARINVS |
| GetARPMTDs |
ARPMTD |
\ARPMTDs |
| GetARPMTs |
ARPMT |
\ARPMTs |
| GetARTRANs |
ARTRAN |
\ARTRANs |
| GetCHARTs |
CHART |
\CHARTs |
| GetCOs |
CO |
\COs |
| GetCOITEMs |
COITEM |
\COITEMs |
| GetCO-BLNs |
CO-BLN |
\CO-BLNs |
| GetCO-SHIPs |
CO-SHIP |
\CO-SHIPs |
| GetCUSTADDRs |
CUSTADDR |
\CUSTADDRs |
| GetCUSTOMERs |
CUSTOMER |
\CUSTOMERs |
| GetDISPCODEs |
DISPCODE |
\DISPCODEs |
| GetEMPLOYEEs |
EMPLOYEE |
\EMPLOYEEs |
| GetEVALCODEs |
EVALCODE |
\EVALCODEs |
| GetEXC-MESGs |
EXC-MESG |
\EXC-MESGs |
| GetFEATQTYs |
FEATQTY |
\FEATQTYs |
| GetFEATRANKs |
FEATRANK |
\FEATRANKs |
| GetFEATUREs |
FEATURE |
\FEATUREs |
| GetFRZCOSTs |
FRZCOST |
\FRZCOSTs |
| GetINV-HDRs |
INV-HDR |
\INV-HDRs |
| GetINV-ITEMs |
INV-ITEM |
\INV-ITEMs |
| GetITEMs |
ITEM |
\ITEMs |
| GetITEMLOCs |
ITEMLOC |
\ITEMLOCs |
| GetITEMPRICEs |
ITEMPRICE |
\ITEMPRICEs |
| GetITEMVENDs |
ITEMVEND |
\ITEMVENDs |
| GetITEMWHSEs |
ITEMWHSE |
\ITEMWHSEs |
| GetJOBs |
JOB |
\JOBs |
| GetJOBITEMs |
JOBITEM |
\JOBITEMs |
| GetJOBMATLs |
JOBMATL |
\JOBMATLs |
| GetJOBROUTEs |
JOBROUTE |
\JOBROUTEs |
| GetJOBTRANs |
JOBTRAN |
\JOBTRANs |
| GetJOB-REFs |
JOB-REF |
\JOB-REFs |
| GetJOB-SCHs |
JOB-SCH |
\JOB-SCHs |
| GetJOURNALs |
JOURNALs |
\JOURNALs |
| GetJRT-SCHs |
JRT-SCH |
\JRT-SCHs |
| GetLEDGERs |
LEDGER |
\LEDGERs |
| GetLOCATIONs |
LOCATION |
\LOCATIONs |
| GetLOTs |
LOT |
\LOTs |
| GetMATLTRACKs |
MATLTRACK |
\MATLTRACKs |
| GetMRP-EXCs |
MRP-EXC |
\MRP-EXCs |
| GetMRP-PARMs |
MRP-PARM |
\MRP-PARMs |
| GetMRPs |
MRP |
\MRPs |
| GetMRP-WBs |
MRP-WB |
\MRP-WBs |
| GetNOTEs |
NOTE |
\NOTEs |
| GetP-FUNCTs |
P-FUNCT |
\P-FUNCTs |
| GetPARMs |
PARM |
\PARMs |
| GetPERMITs |
PERMIT |
\PERMITs |
| GetPOs |
PO |
\POs |
| GetPOBLNCHGs |
POBLNCHG |
\POBLNCHGs |
| GetPOCHANGEs |
POCHANGE |
\POCHANGEs |
| GetPOCHGs |
POCHG |
\POCHGs |
| GetPOITEMs |
POITEM |
\POITEMs |
| GetPO-BLNs |
PO-BLN |
\PO-BLNs |
| GetPRICECODEs |
PRICECODE |
\PRICECODEs |
| GetPROBCODEs |
PROBCODE |
\PROBCODEs |
| GetPRTRXPs |
PRTRXP |
\PRTRXPs |
| GetRCPTs |
RCPT |
\RCPTs |
| GetRMAs |
RMA |
\RMAs |
| GetRMAITEMs |
RMAITEM |
\RMAITEMs |
| GetSERIALs |
SERIAL |
\SERIALs |
| GetSHIPCODEs |
SHIPCODE |
\SHIPCODEs |
| GetSLSMANs |
SLSMAN |
\SLSMANs |
| GetTERMs |
TERM |
\TERMs |
| GetUSER-LOCALs |
USER-LOCAL |
\USER-LOCALs |
| GetVCH-HDRs |
VCH-HDR |
\VCH-HDRs |
| GetVCH-ITEMs |
VCH-ITEM |
\VCH-ITEMs |
| GetVENDADDRs |
VENDADDR |
\VENDADDRs |
| GetVENDORs |
VENDOR |
\VENDORs |
| GetWCs |
WC |
\WCs |
| GetWHSEs |
WHSE |
\WHSEs |
Each DataGrabber includes the Microsoft Visual Basic source code for the DataGrabber which allows you to modify the program
if necessary. This allows you to change your DataGrabbers when the structure of one of your Progress tables
changes. Having the source code will also allow you to add features to the DataGrabber programs such validating
or modifying data as the table is replicated, or moving your Progress tables into programs or databases other than your
Microsoft SQL Server.
Syteview also gives you the tools you need to create new DataGrabber programs allowing you to create
DataGrabbers for your own user-defined Progress tables or for Progress tables used by other programs.
Creating Syteview DataGrabbers
Microsoft SQL Server uses a Windows NT resource called Data Transformation Services (DTS) to work with data services
and structures. Using the Data Transformation Services you can create DTS Packages that will package
together all the steps required to import data from your Progress database into the Microsoft SQL Server Syteline® database.
Microsoft SQL Server provides you with an easy to use GUI program, the DTS Designer, that helps you design and run
these DTS Packages. To create a DTS Package that imports a table from your Progress database make sure your
Progress Database Server is running and then open the SQL Server Enterprise Manager program, expand the SQL Server
Group, expand your Server, then right click Data Transformation Services, select All Tasks and then
select Import Data. This will bring up the following screen.
Click the Next button to proceed.
On the next screen select the MERANT 3.6 32-BIT Progress SQL92 v9.1B ODBC driver which will connect to your Progress
database and retrieve table structures and records. You will need to select the System DSN: Progress-ODBC
and enter your Username and Password before clicking the Next button to proceed.
Note: Leave the Username and Password fields blank if you have not enabled security on your Progress database.
Note: See Creating the Progress-ODBC Connection for instructions
on creating the ODBC connection to your Progress database.
On the next screen select the Microsoft OLE DB Provider for SQL Server driver which will connect to your Microsoft
SQL Server and create table structures and import the records.
Put your SQL Server’s name, or (local), in the Server field. Make sure Use Windows NT Authentication is
selected and put Syteview in the Database field.
Click the Next button to continue.
One this next screen select Copy table(s) and views from the source database.
Click the Next button to continue.
On the next screen select the Progress database table you want to create the DataGrabber for. In this example we
will be creating a DataGrabber for the CO table.
Click the Next button to proceed.
Note: If you see an error message here backup and make sure your Progress Database Server is running.
On the next screen uncheck the Run Immediately checkbox and checkmark the Save DTS Package checkbox.
Click the Next Button to continue.
On this next screen give the DTS Package a name and description. Put your SQL Server’s name in the Server
name field and make sure Use Windows NT authentication is selected.
Click the Next Button to proceed.
The next screen just tells you that you have successfully created a DTS Package that will import the selected table from the
Progress database into the SQL Server.
Congratulations. Click the Finish Button.
You just created a DTS Package. To run the Package and import the table and records from the Progress database go
back to the SQL Server Enterprise Manager and expand Data Transformation Services and click on Local Packages.
Double click the Package you just created to open it.
This is the DTS Package you just created.
Pull down the Package menu and select Execute to run the package and import the records.
Hopefully you will see something like this:
Unfortunately, there is a problem with this DTS Package. The next time you execute it, the package will import the
same records again right on top of the records you just imported. To fix this problem we simply add one more step
to the beginning of the package that drops the table before proceeding.
To do this select the Create Table icon and copy and paste it just to the left of the original Create Table icon.
Double click the new icon and change the Description from Create Table to Drop Table and change the SQL
Statement to DROP TABLE [Syteline].[dbo].[your table name] and click OK.
Finally select the Drop Table icon you just created by clicking on it, then hold down the Ctrl key and click on the
Create Table icon . Make sure you selected the Drop Table icon
BEFORE you select the Create Table icon. Now pull down the Workflow menu
and select On Completion.
You should now see an arrow going from the Drop Table icon to the Create Table icon.
Finally, pull down the Package menu, select Save and you are done.
Done creating the DTS Package that is. Now if you want to be able to create a Visual Basic program (a Syteview
DataGrabber) that executes the same steps you have just specified, run the DTS2VB.exe program in the DTS-to-VBCode
sub-directory.
Question: Why would you want to create a Visual Basic program that grabs tables and records from the Progress database
and puts them in the Microsoft SQL Server when you already have a nice DTS Package that does the same thing?
Well, for one thing, you can use that VB program to create an executable that you can call from a batch file that imports the
table and records at a specified time using Windows Task Scheduler. You could automate the DTS Package using
SQL Server Agents, but by using the Task Scheduler you can change the schedule without opening the SQL Server Enterprise
Manager and you might feel a lot more comfortable letting someone else change the schedule using the Task Scheduler than you
would letting them mess around inside the SQL Server Enterprise Manager.
Another, and maybe a better reason is you can modify the source code of the VB program to do almost anything else you might
want while the records are being imported. Like maybe sending an email when it sees a new customer order, or… well
that’s up to you. When you have the source code you can do almost anything! Or you can just schedule
the DTS Packages and forget the next steps.
If you do want to create a new Data Grabber from the DTS Package then run the DTS2VB.exe program
in the \DTS-to-VBCode sub-directory in the Syteview installation directory on your SQL Server.
The following window will pop up:
Fill in the name of the DTS Package you just created. GetCOs in this case.
And then click OK.
On the next screen, fill in the Progress Username you will use when importing records from the Progress table if you
have enabled security on your Progress database. Leave the Username blank if you have not enabled security on
your Progress database or if you are using a Password file.
On the next screen, fill in the Progress Password you will use when importing records from the Progress table if you
have enabled security on your Progress database. Leave the password blank if you have not enabled security on
your Progress database or if you are using a Password file.
Note: The Password file mentioned above is located in the \DataGrabbers\Password sub-directory and is
named ProgressDB.Password.txt. If this file exists then each DataGrabber in the \DataGrabbers directory
will use the username and password listed within when connecting to the Progress database to retrieve records.
Using this file allows you to change the Progress username and password without having to recompile the Visual Basic code
that makes up your DataGrabbers.
Note: If you have not enabled security on your Progress database then you should rename or delete this file before
creating your DataGrabber programs.
On the next screen, fill in the Microsoft SQL Server Username you will use when connecting to the SQL Server if
you are not using NT Integrated Security. Leave the username blank if you are using NT Integrated
Security.
Note: You really SHOULD be using NT Integrated Security to connect to your SQL Server.
On the next screen, fill in the Microsoft SQL Server Password you will use when connecting to the SQL Server if
you are not using NT Integrated Security. Leave the password blank if you are using NT Integrated
Security.
After clicking OK you will (hopefully) see the following screen:
Assuming you see this screen, you will find the source code for the new program that was just created (in this case it’s
GetCOs.txt) in the \DTS-to-VBCode directory. Highlight the program and right click it and select Open
With. When the Open With window pops up select Visual Basic 6 and click OK.
Note: you might want to move the GetCOs.txt file to its own sub-directory before using it to create the Visual Basic
program. We’ve put each of the GetXXXs programs that come with Syteview in their own sub-directory under the
\DataGrabbers directory.
Visual Basic will open up with the GetCOs.txt in the editor.
Pull down the Project menu and select
References. In the References window checkmark the Microsoft DTSPackage Object Library
Click OK to close the window.
Pull down the File menu and select Save Project As. Give the project a name similar to the
DTS Package name. For example: GetCO.vbp
Next, pull down the File menu again and select Make Exe.
You have just created a new Data Grabber that will import a table and
records from your Progress database and put it in the Microsoft SQL Server. Way to go.
To run the Data Grabber, put it in a sub-directory of its own and copy an INDEXES.TXT and a PERMISSIONS.TXT
file from the sub-directory of a similar Data Grabber. You could use the INDEXES.TXT and PERMISSIONS.TXT files
from the \Syteview_SQL\DataGrabbers\GetCOs sub-directory on the Syteview Installation CD-ROM in this case.
Then edit the Permissions and Indexes files to grant the permissions and create the indexes you want on the table the new
Data Grabber will be creating.
Finally, pop out to the DOS command line, change into the sub-directory with your new Data Grabber in it and run it.
Easy, wasn’t it?
|
|
|