This document provides material to get you started developing server-side web applications using Delphi 3 and the CGI Expert component suite. It also gives information on how to set up the Microsoft Personal Web Server and your browser so that you can test your applications.

Understanding HTTP and server scripts

The hypertext transfer protocol (HTTP) is a bit like a game of tennis. The remote browser generates a bundle of information called a "request" and hits it across the Net to the server. The server looks at the request and then bundles up a "response" and sends it back over the Net to the browser. The game can go on for as long as the browser keeps sending requests.

The most common request is a simple Universal Resource Locator or URL. Most browsers display the URL in an edit box, and it usually looks something like http://hostname.domainname.com/documentname.html

Frequently the URL is just a reference to a document in which case the server simply returns that document as the response. Often the document is HTML, but it doesn't have to be. It could for example be a GIF or JPG graphic file. To allow the browser to determine response types the server attaches extra information to them in the form of "HTTP headers" the detail of which we need not be concerned about just now.

Instead of a document the URL could also be a reference to a server-side web application, or "script". In this case the server invokes the script which then takes responsibility for bundling up the response. Again, the response doesn't have to be just HTML.

[Note: Server-side web applications are called "scripts" from the tradition of writing them in languages like Perl or Bourne Shell. I think the name has stuck because the alternative is a heck of a mouthful]

A script usually needs some sort of input from the browser for it to know what to do. HTTP has a couple of methods of sending parameters to the script. The simplest is URL encoding which involves adding a question mark '?' after the script name followed by a string. For example:

http://cgi-bin/scriptname?parameter1=value1&parameter2=value2&parameter3=value3

The character set that can form a URL is limited so there is a convention for encoding other characters using the percent sign followed by the hex ASCII number of the character. For example a space is represented as %20 and '%' is %25.

A problem with using URL encoding is that anyone can see sensitive "password" type information just by looking over the shoulder of the operator. Another problem is that some browsers will truncate long URLs. On the plus side however, the use of URL encoding means that the operator can "bookmark" the URL for use later, but this can also be a problem if the parameters can go stale over time.

Parameters can come from various sources, but most commonly they come from an HTML FORM. Typically the operator fills in the form and invokes the "submit" button thus creating a request which is sent by the browser.

If the FORM tag contains the words "METHOD=GET" then the browser will use URL encoding to pass the parameters. Alternatively a FORM can have METHOD=POST in which case the browser will send the form parameters over a communications channel that is opened up between the browser and the server. Be aware however, unless you are using a secure channel it will be still be sent as plain text and could be intercepted using a "sniffer".

When the script is invoked through CGI (Common Gateway Interface) the parameters reach it as a combination of environment variables and "standard input". Bob Swart's article "Delphi 2 and CGI" in issue 15 of The Delphi Magazine shows how you can use vanilla Delphi to read this data.

Visual Basic was unable to access standard input so Robert Denny invented WinCGI for O'Reilly's WebSite server and this has become a widely adopted standard. This basically stuffs the form parameters into an INI file rather than use environment variables. This tended to slow things down a bit, but VB users are used to that! Steve Troxell's article "Developing Dynamic Web Pages Part 1" in issue 16 of The Delphi Magazine gives more information on using WinCGI in Delphi.

Another alternative to CGI and WinCGI is Microsoft's ISAPI. With ISAPI you write your "script" as a DLL with specially defined entry points. The request parameters are then passed using memory data structures. ISAPI has the advantage that it is a lot quicker, but the disadvantage that it's very easy for an ISAPI DLL to crash the server because it runs in the same address space.

Steve Troxell covers ISAPI pretty well in another excellent article "Internet Apps With ISAPI" in issue 19 of The Delphi Magazine.

Finally, there is also another API for the Netscape Server called, you guessed it, NSAPI. I don't know much about this but I imagine it is similar to ISAPI.

So as you can see there are lots of different ways in which your script can receive data from the remote browser, and we haven't even looked at how we would send the response from our app back to the server. This varies from writing to "standard output" in CGI through to calling special "write" functions in ISAPI.

Ideally we should be able to write an application that works in all of these environments, or is at least easily modified to do so. You can do this using vanilla Delphi but it is a lot of hard work. Add to this problems of session management, image conversions and e-mailing and this is where a tool like CGI Expert comes in.

CGI Expert is a component suite by Lars Akerman to facilitate development of server-side web applications or "scripts".

It will run on Delphi 3, Delphi 2 and C++ Builder.

The main benefit of CGI Expert is that it manages the HTTP request-response conversation for you independently of the specific interface used, CGI, WinCGI, ISAPI or NSAPI.

The central component is TGeneralHttpEngine that interprets the request and makes it available in an easily accessible form. This includes properties to represent common server variables as well as access to named HTML form variables - regardless of whether the method was GET or POST.

TGeneralHttpEngine also supports the generating of a response and passing it from your application back to the browser. It can automatically generate the necessary HTML headers. There are also a number of filter components that will read HTML templates from strings or files and give your application the opportunity to replace special tags in the templates with dynamic data.

Another key feature is support for session management. This uses cookies to detect when a user is working in a given session and it maintains a set of session variables that you can access to tell you the current state of each session. Pure HTTP is a stateless protocol so you need something like this to be able to manage anything other than the most trivial request responses.

Other features:

Current version of CGI Expert is 3.03B, released earlier this month (Feb 98)

Current cost for shareware versions is $US75 (no source) or $US175 (with source)

Lars also has a usable freeware version, but with a lot fewer features.

Support for the products is by e-mail, and there is also a web-based discussion group.

See http://www.CGI Expert.com for more information.

I found installation of CGI Expert for Delphi 3 to be a breeze. Simply unpack the zip file into a scratch directory and run CGIExpert.exe. This will automatically install all packages, components and help files.

Installation for Delphi 2 and C++ Builder looks a bit more complicated, but the instructions seem clear enough and I'm sure most developers will cope.

This section assumes you have already set up a test environment with a local web server and a browser. If not, please see Configuring your test environment below.

Make a directory for your application and create your Delphi project there. This directory should be under either the scripts directory which is a sibling to the root directory of your web server, or under the cgi-bin directory below the root directory.

I would suggest that under the cgi-bin directory is preferable as it is more like the location in which your application will probably reside on a live web server. This also means that you will be able to include relative links in your response to files in the HTTP readable directory above.

You need to have a form in your project to allow you to debug it, but I suggest that you leave it empty so that you can remove it later if you want to. By removing the form you will make the EXE smaller, you will also be able to more easily convert the app into a DLL if you wish.

Create a data module and add a TGeneralHttpEngine component.

For an ultra simple example put the following line of code in its OnExecRequest:

Put('<HTML><BODY>Hello world');

Compile and run the project.

Open Internet Explorer and request the following URL

http://localhost/cgi-bin/test/project1.exe

Note: This assumes you have put your project in a directory called test under cgi-bin

You should see the Hello world message appear on your browser.

Since you are running the application under the IDE you have all of the usual debugging facilities available.

CGI Expert's TGeneralHttpEngine is similar to the C/S TWebDispatcher with only one allowed per application. TGeneralHttpEngine encapsulates the Http request parameters similar to the C/S TWebRequest.

The C/S TPageProducer is most closely represented in CGI Expert by THttpMemoFilter and THttpFileFilter, along with a number of components for creating and handling HTML Form controls. A particularly nice component is THttpDBSelect which will populate a Select list from the database, a bit like an HTML version of TDBListBox.

Support for HTML tables is not as strong as the C/S web components but the TDataSetTableProducer is at least partly represented by CGI Expert's THttpDBGrid.

Debugging seems much easier with CGI Expert, I don't know why but with C/S it seems a lot harder to set up a debugging session. See pages 34-22 to 34-24 in Ch 34 of the Delphi 3 Developer Guide. Bob Swart went to a lot of trouble to "fake" a CGI environment for his CGI debugger (see his "Under Construction" article in Delphi Magazine Issue 19) With CGI Expert you don't seem to have to go to all this trouble.

To debug with CGI Expert you just compile your app in an http-visible directory (eg: below cgi-bin) and run it using the Delphi IDE. You can then send requests to your app using your favourite browser. You can trap breakpoints, single-step etcetera, but be aware that if your app does not respond quickly the HTTP connection will time out and you will see an "incomplete headers" message on your browser.

CGI Expert doesn't have the sophisticated action dispatching mechanism of C/S. I think this makes CGI Expert easier to understand, but probably limits the complexity of problems it can deal with.

You need to set up TCP/IP on your system. If you are already accessing the Internet you will probably have already done this. If not then you should be able to install TCP/IP as follows:

1. Go to Control Panel->Network (can be reached via the Settings item in the Start Menu)

2. Click "Add" and select "Protocol".

3. Choose "Microsoft" and then "TCP/IP".

This document assumes you already have your browser installed, but there are still a couple of things you may have to attend to.

If you have a dial in connection to the Internet you may see the dialler box appear when you start your browser, you can choose to continue and dial up your ISP, but as you will be using a local web server you can just cancel the dial up.

If this annoys you then you can switch it off. For Internet Explorer 3.02 go to the Connections page in the Options dialog and un-check "Connect to the Internet as needed" thus:

IMPORTANT NOTE ON PROXY SERVERS: Many ISPs now offer a proxy service, and if yours does then you will probably find that "Connect through proxy server" is checked in your browser options. If so then you will not be able to open localhost unless you actually dial up your ISP. To allow you to access your local web server off line you will have to un-check "Connect through proxy server" as shown above. You will then need to remember to go back and check it before you can use your ISP once again.

To test your dynamic web applications you will need to set up a local web server. There are several server options, but I use the Microsoft Personal Web server (PWS) that comes "free" with later versions of Windows 95.

But first be warned that if you are on a local are network and use password authenticated file sharing, otherwise known as "Share level access control" PWS will change this to "User level access control". Look for Control panel->Network->Access Control. If you don't see this page then this isn't a problem for you as you aren't on a network. It also isn't a problem if you don't share your drives or printers, or if you are happy with authentication by user rather than by password.

PWS isn't particularly well documented but there is a file PWS.TXT in your Windows 95 directory that is worth a look. The following is an extract.

To install the Personal Web Server, go to Network in the Control Panel and press Add, select Service, press Add, select Microsoft from the Manufacturers column, double click on Personal Web Server, and press OK on the network control panel.

Note that you will need your Windows 95 CD-ROM (I am using version OSR2) The server will be installed in C:\Program files and you don't seem to have a choice about this.

PWS.TXT then goes on to say.

By default your home page is C:\Webshare\WWWroot\DEFAULT.HTM. You can edit this file to fit your needs.

If you have MS Office 97 then you can open and edit DEFAULT.HTM in Word. You will be asked if you want to load the latest Internet Assistant from the Web. This isn't really necessary and can take a lot of time.

You can get by with Word 97 and Internet Assistant, but it's a bit flakey and I prefer to use MS Frontpage. It has a lot of fancy site management features which are actually quite useful, but it's worth getting Frontpage for the solid WYSIWYG HTML editor alone.

If you haven't got Office 97, Frontpage or any other HTML editor you can edit HTML in any text editor, but it's not very pretty.

You can test that the web server is running by opening up your browser and giving localhost as the URL. I use Internet Explorer 3.02 and it looks like this:

There's no need to type the http:// in front of localhost. Explorer will automatically add that for you.

You are supposed to be able to control PWS via the Personal Web Server shortcut in Control Panel, or by right-clicking the PWS icon on the task bar. However, I found that this doesn't always work. If you find you have problems connecting to the admin page, fire up Internet Explorer and give the following URL: http://localhost/HtmlaScripts/htmla.dll?http/serv

Having got your PWS admin page up and running, I would recommend you do the following:

1. Switch off "Allow anonymous". This will stop people remotely accessing the the web applications that you are debugging. Of course this can only happen while you are actually connected to a TCP/IP network.

2. Create a directory in which to run your "scripts" called cgi-bin in your server's root directory (usually C:\WebShare\wwwroot). In the PWS admin page make it a "Virtual directory" of the same name i.e cgi-bin and set its access to be "Execute" instead of "Read". You could leave the "Read" box checked but this will not be how it is usually set up on a "live" web server. NOTE: If you install FrontPage it will automatically create a cgi-bin for you, as well as other directories that it uses.

A bit more information on installing PWS can be found at http://www.futurenet.com/pcplus/IE4cd/pws.htm

Under Construction: Delphi 2 and CGI
The Delphi Magazine. Issue #15 (November 1996), Page #25
Author: Bob Swart (aka Dr.Bob)
Summary: Bob Swart develops an interactive World Wide Web based version of The Delphi Magazine Book Review Database, showing that developing applications for the Internet is nothing to be scared of!

Developing Dynamic Web Pages
The Delphi Magazine. Issue #16 (December 1996), Page #8
Author: Steve Troxell
Summary: Steve takes us deeper into the creation of Web applications, developing a CGI component that works with both conventional CGI and WinCGI.

Note: The above reference is very good at explaining CGI/WinCGI and ISAPI and how to write a CGI app in vanilla Delphi. Steve also gives us a component for parsing the CGI input.

Developing Dynamic Web Pages: Part 2
The Delphi Magazine. Issue #17 (January 1997), Page #18
Author: Steve Troxell
Summary: Steve describes how to add database support to your Web apps, using TDMAid Online, our own web-based article index database, as an example.

Under Construction: Delphi 2 and CGI
The Delphi Magazine. Issue #15 (November 1996), Page #25
Author: Bob Swart (aka Dr.Bob)
Summary: Bob Swart develops an interactive World Wide Web based version of The Delphi Magazine Book Review Database, showing that developing applications for the Internet is nothing to be scared of!

Internet Apps With ISAPI
The Delphi Magazine. Issue #19 (March 1997), Page #28
Author: Steve Troxell
Summary: Steve Troxell demonstrates how to use the API for Microsoft's IIS web server to good effect for go-faster internet apps

Under Construction: Delphi 3 Web Modules, Part 1
The Delphi Magazine. Issue #24 (August 1997), Page #8
Author: Bob Swart (aka Dr.Bob)
Summary: Bob Swart begins a 2-part investigation into the new web modules included with Delphi 3's Client/Server Edition

Web Commerce With Delphi
The Delphi Magazine. Type: Article, Issue #25 (September 1997), Page #26
Author: Peter Hyde
Summary: As the web gets more and more popular, so companies are more interested in using it to sell their wares but how can you build the secure web apps needed? Peter Hyde discusses the issues and pitfalls and demonstrates a simple online sales application created using the WebHub application framework

Under Construction: Delphi 3 Web Modules, Part 2
The Delphi Magazine. Issue #25 (September 1997), Page #40
Author: Bob Swart (aka Dr.Bob)
Summary: Continuing on from last month, Bob Swart shows how to write dynamic queries, save state information and even eat some cookies along the way...

Under Construction: Homepage Hit Counter
The Delphi Magazine. Issue #26 (October 1997), Page #28
Author: Bob Swart (aka Dr.Bob)
Summary: Who's been coming to call? Bob Swart shows how to construct various types of hit counter for your website so you can discover exactly how popular you are!

Get On The Web With Delphi 3
The Delphi Magazine. Issue #28 (December 1997), Page #35
Author: John O'Connell
Summary: In a two-part series, John O'Connell aims to explain (nearly) everything you need to know about writing web applications but didn't know you needed to ask

Get On The Web With Delphi 3 Part 2
The Delphi Magazine. Issue #29 (January 1998), Page #37
Author: John O'Connell
Summary: John O'Connell presents the concluding part of his discussion of "everything you need to know about writing web applications with Delphi 3 (but didn't know you needed to ask)"

Ceating Internet Server Applications
Borland Delphi 3 Developers Guide (comes with D3)
Publisher: Borland
See Chapter 34 "Creating Internet Server Applicatons"
Note: This focuses mainly on the C/S web module components, but also gives good background on how server applications work.

Web Publishing Unleashed
Editor: William R. Stanek
Publisher: Sams.net Publishing
Note: This book is easy to read and contains lots of information, but suffers a bit from being a "compilation" of work by several authors.

How to Set Up and Maintain a Web Site (2^nd edn)
Author: Lincoln D. Stein
Publisher: Addison Wesley
Note: Chapter 8 "Working with Server Scripts" is excellent, although it's a bit Unix oriented.