summary |
shortlog |
log |
commit | commitdiff |
tree
raw |
patch |
inline | side by side (from parent 1:
09a776a)
Twitter.com has started serving the user timeline via json when the user
agent is a modern browser, this breaks scraping in Tweeper which expects
html content.
Remove any version info from the User-Agent header used by Tweeper to
make twitter.com think it is talking with a very old browser, tricking
it into serving html content.
NOTE: Tweeper cannot just use the default User-Agent from the CURL
library because this would break scraping Facebook.com; using a minimal
but still browser-like User-Agent seems to be a viable common
denominator for all sites currently supported by Tweeper.
- private static $userAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:64.0) Gecko/20100101 Firefox/64.0";
+ private static $userAgent = "Mozilla/5.0";
private static $maxConnectionTimeout = 5;
private static $maxConnectionRetries = 5;
private static $maxConnectionTimeout = 5;
private static $maxConnectionRetries = 5;