If you get "Connection reset by peer" while scraping a website, it is very likely that your scraping attempts were detected and automatically blocked for a while. They might have also been alerted by all of these malformed requests.
Consider pacing out your requests so that it doesn't look like you are scraping.
--
On Monday, December 31, 2018 at 7:25:13 AM UTC-5, Colin Law wrote:On Mon, 31 Dec 2018 at 12:12, fugee ohu <fuge...@gmail.com> wrote:
>
>
>
> On Sunday, December 30, 2018 at 10:56:30 AM UTC-5, Colin Law wrote:
>>
>> On Sun, 30 Dec 2018 at 15:45, fugee ohu <fuge...@gmail.com> wrote:
>> >
>> >
>> >
>> > On Sunday, December 30, 2018 at 8:36:40 AM UTC-5, Colin Law wrote:
>> >>
>> >> On Sun, 30 Dec 2018 at 12:06, fugee ohu <fuge...@gmail.com> wrote:
>> >> > ...
>> >> > It's not json it's javascript so I don't have to run JSON.parse
>> >>
>> >> {"success":true,"code":0,"results":[{"productId":32815555905, ...
>> >> Looks like JSON to me (embedded in js admittedly). You said that the
>> >> data you want is in that string. If that is correct then all you have
>> >> to do is to extract it and parse it as JSON.
>> >>
>> >> Colin
>> >
>> >
>> > It's not json it's javascript
>>
>> What is it about the string
>> {"success":true,"code":0,"results":[{"productId":32815555905, ...
>> that makes it not JSON?
>>
>> Colin
>
>
> Now I only get connection reset by peer when I try to make the request
Can't help you there, presumably either the website has changed or you
have changed the way you are fetching it. Try the url in a browser.
ColinI changed my code and now getting a text/html content type response Not sure what I'm doing I commented out my previous creation of http object and used an inline syntax that's part of the creation of res objectrequire "net/http"require "uri"url = URI.parse("https://www.ali<notshown>.com/item/Robotic-Vacuum-Cleaner-Proscenic-790T-Vacuum-Mop-Sweep-3-in-1-Cleaner-for-Pet-Hair-Wifi/32840149410.html?spm=2114.search0104.3.1.24d566b6GAD2uI&ws_ab_test=searchweb0_0,searchweb201602_1_10065_10068_10130_10890_10547_319_10546_317_10548_5730311_10545_10696_453_10084_454_10083_5729211_10618_10307_538_537_536_10059_10884_10887_100031_321_322_10103-10890,searchweb201603_51,ppcSwitch_0&algo_expid=99dc32b9-d1ce-4020-8bec-624c18225f44-0&algo_pvid=99dc32b9-d1ce-4020-8bec-624c18225f44")#http = Net::HTTP.new(url.host, url.port)#http.use_ssl = truereq = Net::HTTP::Get.new urlres = Net::HTTP.start(url.host, url.port, :use_ssl => url.scheme == 'https') {|http| http.request req}puts res.body
You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rubyonrails-talk+unsubscribe@googlegroups.com.
To post to this group, send email to rubyonrails-talk@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/rubyonrails-talk/638bfc50-c217-4a52-892e-a3302b537f47%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rubyonrails-talk+unsubscribe@googlegroups.com.
To post to this group, send email to rubyonrails-talk@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/rubyonrails-talk/CALn2xuD6pOYcjGXwnqJ_w6iDAOcAgvZVd4e6uAAwg9-Yf9GY1g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rubyonrails-talk+unsubscribe@googlegroups.com.
To post to this group, send email to rubyonrails-talk@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/rubyonrails-talk/CAJR%2B9kbDouvoG_pPtNbVcKrOdZFB116CiK5KLjfPGqKx_tVxPA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Steps to reproduce
- Create Rails 6 app
- yarn add onsenui
- Add below lines to "app/assets/stylesheets/application.css"
*= require onsenui/css/onsenui *= require onsenui/css/onsen-css-components - Create a sample view as below
<ons-page> <div style="text-align: center; margin-top: 30px;"> <p> <ons-input id="username" modifier="underbar" placeholder="Username" float></ons-input> </p> <p> <ons-input id="password" modifier="underbar" type="password" placeholder="Password" float></ons-input> </p> <p style="margin-top: 30px;"> <ons-button onclick="login()">Sign in</ons-button> </p> </div> </ons-page> - Access that page
Expected behavior
Page should render without error.
Actual behavior
$ rails s
=> Booting Puma
=> Rails 6.0.0.alpha application starting in development
=> Runrails server --helpfor more startup options
Puma starting in single mode...
- Version 3.12.0 (ruby 2.5.3-p105), codename: Llamas in Pajamas
- Min threads: 5, max threads: 5
- Environment: development
- Listening on tcp://0.0.0.0:3000
Use Ctrl-C to stop
Started GET "/pages/auth" for 127.0.0.1 at 2019-01-02 12:17:55 +0530
(5.7ms) SELECT sqlite_version(*)
Processing by PagesController#auth as HTML
Rendering pages/auth.html.erb within layouts/application
Rendered pages/auth.html.erb within layouts/application (Duration: 2.9ms | Allocations: 350)
Completed 500 Internal Server Error in 377ms (ActiveRecord: 0.0ms | Allocations: 287951)ActionView::Template::Error (couldn't find file 'onsenui/css/onsenui' with type 'text/css'
Checked in these paths:
/home/smitha/Documents/kiran/fooapp/app/assets/config
/home/smitha/Documents/kiran/fooapp/app/assets/images
/home/smitha/Documents/kiran/fooapp/app/assets/stylesheets
/home/smitha/Documents/kiran/rails/actioncable/app/assets/javascripts
/home/smitha/Documents/kiran/rails/activestorage/app/assets/javascripts
/home/smitha/Documents/kiran/rails/actionview/app/assets/javascripts
/home/smitha/.rvm/gems/ruby-2.5.3/gems/turbolinks-source-5.2.0/lib/assets/javascripts):
5: <%= csrf_meta_tags %>
6: <%= csp_meta_tag %>
7:
8: <%= stylesheet_link_tag 'application', media: 'all', 'data-turbolinks-track': 'reload' %>
9: <%= javascript_pack_tag 'application', 'data-turbolinks-track': 'reload' %>
10:
11:app/assets/stylesheets/application.css:14
app/views/layouts/application.html.erb:8
You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rubyonrails-talk+unsubscribe@googlegroups.com.
To post to this group, send email to rubyonrails-talk@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/rubyonrails-talk/38cb0ca0-ce2b-4795-99ba-4706355fd6c4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.