Wget ошибка аутентификации пользователя пароля

I’m at a total loss as to why I’m getting Authorization failed messages when I try to perform basic HTTP auth using wget on an internal webpage.

I’ve been using this wget command to pull a csv file from an internal webpage for around 4 months on a daily basis without issues.

wget --user username --password "password" http://internalwebsite.domain.com/data.csv

The site is not SSL and does prompt for credentials when visiting it with a web browser. I’ve verified that the credentials I am using in the call work in a browser to download the CSV.

I’ve tried modifying the wget command by adding the following additional flags:

--auth-no-challenge

I also changed the —username to —http-user and the password to —http-passwd but those options do not work either.

When I attempt the connection with wget I get a 401 Unauthorized Authorization failed message.

The site is sharepoint,and unless something recently changed this month with IIS that disables calls from wget, I cannot figure out for the life of me what changed…

Any help would be appreciated.

I’m at a total loss as to why I’m getting Authorization failed messages when I try to perform basic HTTP auth using wget on an internal webpage.

I’ve been using this wget command to pull a csv file from an internal webpage for around 4 months on a daily basis without issues.

wget --user username --password "password" http://internalwebsite.domain.com/data.csv

The site is not SSL and does prompt for credentials when visiting it with a web browser. I’ve verified that the credentials I am using in the call work in a browser to download the CSV.

I’ve tried modifying the wget command by adding the following additional flags:

--auth-no-challenge

I also changed the —username to —http-user and the password to —http-passwd but those options do not work either.

When I attempt the connection with wget I get a 401 Unauthorized Authorization failed message.

The site is sharepoint,and unless something recently changed this month with IIS that disables calls from wget, I cannot figure out for the life of me what changed…

Any help would be appreciated.

First off, I am not a Linux guy, but I have to pretend to be one sometimes as a web developer.

So, we have a Red Hat server and I’m using wget in crontab to run some PHP scripts.

We’ve been doing this for some time now and it’s been working fine.

I tried to add another script using wget to run a PHP script behind HTTP authentication. However, despite the fact that the URL works fine and the username and password are correct, we are getting Connection Timed Out errors each time.

What might cause wget to work for unauthenticated URLs, but not authenticated ones?

I’ve tried —user=/—password=, —http-user=/—http-password and Username:Password@ in the URL and all three fail the same way.

Here’s the command in question:

[blahblah user]# wget -t 5 -O /dev/null 'http://Username:Password1!@test.example.com/sub/dir/file-name.php'
--2010-07-07 10:11:55--  http://Username:*password*@test.example.com/sub/dir/file-name.php
Resolving test.example.com... 000.000.000.000
Connecting to test.example.com|000.000.000.000|:80... failed: Connection timed out.
Retrying.

[Repeat ad nauseum]

Any thoughts? Again, wget works, the file with authentication works, but wget calling the file with authentication does not work.

UPDATE: Actually, I get the same timeout if I access the authenticated URL without authentication. Could that mean that Apache is rejecting wget requests for authentication outright? (I’m really treading into speculation territory here. I know almost nothing about Apache configuration.)

First off, I am not a Linux guy, but I have to pretend to be one sometimes as a web developer.

So, we have a Red Hat server and I’m using wget in crontab to run some PHP scripts.

We’ve been doing this for some time now and it’s been working fine.

I tried to add another script using wget to run a PHP script behind HTTP authentication. However, despite the fact that the URL works fine and the username and password are correct, we are getting Connection Timed Out errors each time.

What might cause wget to work for unauthenticated URLs, but not authenticated ones?

I’ve tried —user=/—password=, —http-user=/—http-password and Username:Password@ in the URL and all three fail the same way.

Here’s the command in question:

[blahblah user]# wget -t 5 -O /dev/null 'http://Username:Password1!@test.example.com/sub/dir/file-name.php'
--2010-07-07 10:11:55--  http://Username:*password*@test.example.com/sub/dir/file-name.php
Resolving test.example.com... 000.000.000.000
Connecting to test.example.com|000.000.000.000|:80... failed: Connection timed out.
Retrying.

[Repeat ad nauseum]

Any thoughts? Again, wget works, the file with authentication works, but wget calling the file with authentication does not work.

UPDATE: Actually, I get the same timeout if I access the authenticated URL without authentication. Could that mean that Apache is rejecting wget requests for authentication outright? (I’m really treading into speculation territory here. I know almost nothing about Apache configuration.)

Why are you playing around with wget? Better use some headless browser to automate this task.

What is a headless browser, you ask?

A headless browser is a web browser without a graphical user interface.
They provide automated control of a web page in an environment similar to popular web browsers, but are executed via a command line interface or using network communication.

Two popular headless browsers are phantomjs (javascript) and Ghost.py (python).

Solution using phantomjs

First you will need to install phantomjs. On Ubuntu based systems, you can install it using the package manager or you could build it from source from their home page.

sudo apt-get install phantomjs

After this you write javascript script and run it using phantomjs:

phantomjs script.js

That’s it.

Now, to learn how to implement it for your case, head over to its quickstart guide. As an example, to login to facebook automatically, and take a snapshot, one could use the gist provided here:

// This code login's to your facebook account and takes snap shot of it.
var page = require('webpage').create();
var fillLoginInfo = function(){
var frm = document.getElementById("login_form");
frm.elements["email"].value = 'your fb email/username';
frm.elements["pass"].value = 'password';
frm.submit();
}
page.onLoadFinished = function(){
if(page.title == "Welcome to Facebook - Log In, Sign Up or Learn More"){
page.evaluate(fillLoginInfo);
return;
}
else
page.render('./screens/some.png');
console.log("completed");
phantom.exit();
}
page.open('https://www.facebook.com/');

Look around the documentation to implement it for your specific case. If you face some troubles for your https website due to ssl errors, run your script like this:

phantomjs --ssl-protocol=any script.js

Solution using Ghost.py

To install Ghost.py, you will need pip:

sudo apt-get install python-pip   #On a Debian based system
sudo pip install Ghost.py

Now you have installed Ghost.py. Now, to use it inside a python script, just follow the documentation given in its home page. I’ve tried using Ghost.py on an https website but it somehow didn’t work for me. Do try it and see if it works.

UPDATE : GUI based solution

You can also use tools like Selenium to automate the login process and retrieve the information. It is pretty easy to use. You will just need to install a plugin for your browser from here. And then you can record your process and replay it later on.

Вопрос:

Лучший ответ:

Ответ №1

--password=PASS
--user=USERNAME

Ответ №2

wget http://username:password@example.org/url/
wget --http-user=user --http-password=password http://example.org/url/

Ответ №3

#!/bin/sh

# get the login page to get the hidden field data
wget -a log.txt -O loginpage.html http://foobar/default.aspx
hiddendata=`grep value < loginpage.html | grep foobarhidden | tr '=' ' ' | awk '{print $9}' | sed s/"//g`
rm loginpage.html

# login into the page and save the cookies
postData=user=fakeuser'&'pw=password'&'foobarhidden=${hiddendata}
wget -a log.txt -O /dev/null --post-data ${postData} --keep-session-cookies --save-cookies cookies.txt http://foobar/default.aspx

# get the page your after
wget -a log.txt -O results.html --load-cookies cookies.txt http://foobar/lister.aspx?id=42
rm cookies.txt

Ответ №4

используйте параметры --user=X --password=Y, чтобы указать имя пользователя X и пароль Y.