Newbie networking - just dl'ing data from an URL...

Posts: 713
Joined: 2002.04
Post: #1
Okay, I'm a total newbie to network coding, and I'm looking to begin to dispel my ignorance; I'm currently looking to code a very simple app to gain the beginnings of a familiarity with network coding: a simplified text-based browser.

All I want to be able to code at this stage is a C app which can take an arbitrary URL, download the page at that URL (into a file or preferably into a NULL-terminated string), and display that page sans tags, with any links within that page numbered for easy text-based navigation.

I've done a few searches, from which it looks like the options open to me are: sockets, cURL, and Wget; of those, cURL looks like the easiest of the options to begin with, so...

Am I on the right track?

Can cURL be linked in a C app, or will I need to call it via system()?

Mark Bishop
Student and freelance OS X & iOS developer
Quote this message in a reply
Posts: 39
Joined: 2002.04
Post: #2
curl should certainly be able to do everything you'll need.
I've used it a bit myself to implement https file uploads and downloads with custom certificates. It can also do other interesting stuff, like acting as html form being submitted, check what kind of server you're talking to e.g.

> curl --head

HTTP/1.1 200 OK
Age: 543
X-Cache-TTL: 57
Date: Wed, 18 Apr 2007 21:02:39 GMT
Content-Length: 34373
Content-Type: text/html; charset=utf-8
Expires: Wed, 18 Apr 2007 21:12:39 GMT
Cache-Control: max-age=600
Server: Apache/1.3.33 (Darwin) PHP/4.3.10
X-Cached-Time: Wed, 18 Apr 2007 21:11:07 GMT

as well as basically any other action that you may want to do over one of the common network protocols ((HTTP, HTTPS, FTP, FTPS, GOPHER, DICT, TELNET, LDAP or FILE).

It appears that you can use C.

you might want to check your man pages, for example:

> man curl
> man libcurl
> man libcurl-easy

> man 3 curl_easy_init
> man 3 curl_easy_cleanup
> man 3 curl_easy_setopt
> man 3 curl_easy_perform
Quote this message in a reply
Posts: 1,403
Joined: 2005.07
Post: #3
I second curl, its easy to use and very good

edit: use libcurl instead of system

Sir, e^iπ + 1 = 0, hence God exists; reply!
Quote this message in a reply
Posts: 5,143
Joined: 2002.04
Post: #4
10.4 comes with libcurl; earlier versions you'll have to download and compile it yourself.
Quote this message in a reply
Posts: 713
Joined: 2002.04
Post: #5
Thanks for the suggestions! Grin I've now got a little libcURL app running on 10.4, after much aggravation with the linker Grin

Mark Bishop
Student and freelance OS X & iOS developer
Quote this message in a reply
Post Reply 

Possibly Related Threads...
Thread: Author Replies: Views: Last Post
  Various BSD Networking Questions wyrmmage 25 16,514 Jul 30, 2007 05:09 AM
Last Post: unknown
  Good networking library Ummon 5 6,309 Jan 22, 2007 11:59 PM
Last Post: akb825
  Simple networking? CarbonX 13 11,160 Apr 7, 2005 12:36 AM
Last Post: Andrew
  Saving a Place for Networking Roosterhouse 3 5,484 Aug 6, 2004 02:05 PM
Last Post: FCCovett
  Networking for Multiplayer Games Where to start? NYGhost 2 4,677 Jan 28, 2004 09:15 AM
Last Post: Skorche