Newbie networking - just dl'ing data from an URL...

Moderator
Posts: 698
Joined: 2002.04
Post: #1
Okay, I'm a total newbie to network coding, and I'm looking to begin to dispel my ignorance; I'm currently looking to code a very simple app to gain the beginnings of a familiarity with network coding: a simplified text-based browser.

All I want to be able to code at this stage is a C app which can take an arbitrary URL, download the page at that URL (into a file or preferably into a NULL-terminated string), and display that page sans tags, with any links within that page numbered for easy text-based navigation.

I've done a few searches, from which it looks like the options open to me are: sockets, cURL, and Wget; of those, cURL looks like the easiest of the options to begin with, so...

Am I on the right track?

Can cURL be linked in a C app, or will I need to call it via system()?

Mark Bishop
--
Student and freelance OS X & iOS developer
Quote this message in a reply
Member
Posts: 39
Joined: 2002.04
Post: #2
curl should certainly be able to do everything you'll need.
I've used it a bit myself to implement https file uploads and downloads with custom certificates. It can also do other interesting stuff, like acting as html form being submitted, check what kind of server you're talking to e.g.

> curl --head http://www.apple.com

HTTP/1.1 200 OK
Age: 543
X-Cache-TTL: 57
Date: Wed, 18 Apr 2007 21:02:39 GMT
Content-Length: 34373
Content-Type: text/html; charset=utf-8
Expires: Wed, 18 Apr 2007 21:12:39 GMT
Cache-Control: max-age=600
Server: Apache/1.3.33 (Darwin) PHP/4.3.10
X-Cached-Time: Wed, 18 Apr 2007 21:11:07 GMT

as well as basically any other action that you may want to do over one of the common network protocols ((HTTP, HTTPS, FTP, FTPS, GOPHER, DICT, TELNET, LDAP or FILE).


It appears that you can use C.

you might want to check your man pages, for example:

> man curl
> man libcurl
> man libcurl-easy

> man 3 curl_easy_init
> man 3 curl_easy_cleanup
> man 3 curl_easy_setopt
> man 3 curl_easy_perform
Quote this message in a reply
Sage
Posts: 1,403
Joined: 2005.07
Post: #3
I second curl, its easy to use and very good http://curl.haxx.se/

edit: use libcurl instead of system

Sir, e^iπ + 1 = 0, hence God exists; reply!
Quote this message in a reply
Luminary
Posts: 5,143
Joined: 2002.04
Post: #4
10.4 comes with libcurl; earlier versions you'll have to download and compile it yourself.
Quote this message in a reply
Moderator
Posts: 698
Joined: 2002.04
Post: #5
Thanks for the suggestions! Grin I've now got a little libcURL app running on 10.4, after much aggravation with the linker Grin

Mark Bishop
--
Student and freelance OS X & iOS developer
Quote this message in a reply
Post Reply 

Possibly Related Threads...
Thread: Author Replies: Views: Last Post
  Various BSD Networking Questions wyrmmage 25 10,880 Jul 30, 2007 05:09 AM
Last Post: unknown
  Good networking library Ummon 5 4,536 Jan 22, 2007 11:59 PM
Last Post: akb825
  Simple networking? CarbonX 13 7,268 Apr 7, 2005 12:36 AM
Last Post: Andrew
  Saving a Place for Networking Roosterhouse 3 4,039 Aug 6, 2004 02:05 PM
Last Post: FCCovett
  Networking for Multiplayer Games Where to start? NYGhost 2 3,500 Jan 28, 2004 09:15 AM
Last Post: Skorche