Awesome

ronin-support-web

Description

ronin-support-web is a web support library for ronin-rb. ronin-support-web provides many helper methods for parsing HTML/XML, fetching web pages, and WebSockets.

Features

Provides helper methods for parsing HTML/XML.
- Also provides additional extensions to Nokogiri using nokogiri-ext.
Provides helper methods for working with WebSockets.

Examples

require 'ronin/support/web'
include Ronin::Support::Web

html_parse "<html>...</html>"
# => #<Nokogiri::HTML::Document:...>

HTML

Parse an HTML string:

doc = html_parse("<html>\n  <body>\n    <p>Hello world</p>\n  </body>\n</html>\n")
# => 
# #(Document:0x6ab8 {
#   name = "document",
#   children = [
#     #(DTD:0x6be4 { name = "html" }),
#     #(Element:0x6cd4 {
#       name = "html",
#       children = [
#         #(Text "\n  "),
#         #(Element:0x6e64 {
#           name = "body",
#           children = [
#             #(Text "\n    "),
#             #(Element:0x6ff4 { name = "p", children = [ #(Text "Hello world")] }),
#             #(Text "\n  ")]
#           }),
#         #(Text "\n")]
#       })]
#   })

Parse a HTML file:

doc = html_open("index.html")
# => #<Nokogiri::HTML::Document:...>

Searching an HTML document using XPath or CSS-path:

nodes = doc.search('//div/p')
nodes = doc.search('div p.class')
# => [#<Nokogiri::HTML::Element:...>, ...]

node = doc.at('#id')
# => #<Nokogiri::HTML::Element:...>

Build a HTML document:

doc = html_build do
  html {
    head {
      script(type: 'text/javascript', src: 'redirect.js')
    }
  }
end

puts doc.to_html
# <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "http://www.w3.org/TR/REC-html40/loose.dtd">
# <html><head><script src="redirect.js" type="text/javascript"></script></head></html>

XML

Parse an XML response body:

xml_parse("<?xml version=\"1.0\"?>\n<users>\n  <user>\n    <name>admin</name>\n    <password>0mni</password>\n  <user>\n</users>\n")
# =>
# #(Document:0xdebc {
#   name = "document",
#   children = [
#     #(Element:0xdfe8 {
#       name = "users",
#       children = [
#         #(Text "\n  "),
#         #(Element:0xe178 {
#           name = "user",
#           children = [
#             #(Text "\n    "),
#             #(Element:0xe308 { name = "name", children = [ #(Text "admin")] }),
#             #(Text "\n    "),
#             #(Element:0xe538 { name = "password", children = [ #(Text "0mni")] }),
#             #(Text "\n  "),
#             #(Element:0xe768 { name = "user", children = [ #(Text "\n")] }),
#             #(Text "\n")]
#           })]
#       })]
#   })

Parse a XML file:

doc = html_open("data.xml")
# => #<Nokogiri:XML:::Document:...>

Searching an XML document using XPath:

users = doc.search('//user')
# => [#<Nokogiri::XML::Element:...>, ...]

admin = doc.at('//user[@name="admin"]')
# => #<Nokogiri::XML::Element:...>

Build a XML document:

doc = xml_build do
  playlist {
    mp3 {
      file { text('02 THE WAIT.mp3') }
      artist { text('Evil Nine') }
      track { text('The Wait feat David Autokratz') }
      duration { text('1000000000') }
    }
  }
end

puts doc.to_xml
# <?xml version="1.0"?>
# <playlist>
#   <mp3>
#     <file>02 THE WAIT.mp3</file>
#     <artist>Evil Nine</artist>
#     <track>The Wait feat David Autokratz</track>
#     <duration>1000000000</duration>
#   </mp3>
# </playlist>

Web Requests

Gets a URL and follows any redirects:

get 'https://example.com/'
# => #<Net::HTTPResponse:...>

Gets a URL and parses the HTML response:

get_html 'https://example.com/'
# => #<Nokogiri::HTML::Document:...>

Gets a URL and parses the XML response:

get_xml 'https://example.com/sitemap.xml'
# => #<Nokogiri::XML::Document:...>

Gets a URL and parses the JSON response:

get_json 'https://example.com/api/endpoint.json'
# => {...}

POSTs to a URL and follows any redirects:

post 'https://example.com/form', form_data: {'foo' => 'bar'}
# => #<Net::HTTPResponse:...>

POSTs to a URL and parses the HTML response:

post_html 'https://example.com/form', form_data: {'foo' => 'bar'}
# => #<Nokogiri::HTML::Document:...>

POSTs to a URL and parses the XML response:

post_xml 'https://example.com/form', form_data: {'foo' => 'bar'}
# => #<Nokogiri::XML::Document:...>

POSTs to a URL and parses the JSON response:

post_json 'https://example.com/api/endpoint.json', json: {foo: 'bar'}
# => {...}

WebSockets

Connecting to a WebSocket:

websocket = websocket_connect('ws://websocket-echo.com')

websocket.send_frame("foo bar")
# => 13
websocket.recv_frame
# => <WebSocket::Frame::Incoming::Client:0x000227a4 @decoded=true, @code=nil, @data="foo bar", @version=13, @handler=#<WebSocket::Frame::Handler::Handler07:0x00007f7d4c9b0ed0 @frame=<WebSocket::Frame::Incoming::Client:0x000227a4 @decoded=true, @code=nil, @data="foo bar", @version=13, @handler=#<WebSocket::Frame::Handler::Handler07:0x00007f7d4c9b0ed0 ...>, @type=:text>, @application_data_buffer=nil>, @type=:text>
websocket.send_frame("hello world")
# => 17
websocket.recv
# => "hello world"

Starting a WebSocket server and receiving connections:

server = websocket_server('ws://localhost:1337/')
client = server.accept
client.send("hello")
client.recv
# => "good, how are you"

client = websocket_client('ws://localhost:1337/')
client.recv
# => "hello"
client.send("good, how are you")

Requirements

Ruby >= 3.0.0
ronin-support ~> 1.1
nokogiri ~> 1.4
nokogiri-ext ~> 0.1
websocket ~> 1.2

Install

$ gem install ronin-support-web

Gemfile

gem 'ronin-support-web', '~> 0.1'

gemspec

gem.add_dependency 'ronin-support-web', '~> 0.1'

Development

Fork It!
Clone It!
cd ronin-support-web/
bundle install
git checkout -b my_feature
Code It!
bundle exec rake spec
git push origin my_feature

License

ronin-support-web - A web support library for ronin-rb.

ronin-support-web is free software: you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

ronin-support-web is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details.

You should have received a copy of the GNU Lesser General Public License along with ronin-support-web. If not, see https://www.gnu.org/licenses/.