Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
61 commits
Select commit Hold shift + click to select a range
3250d30
Add vendor in gitignore
mosharaf13 May 29, 2023
ccfe931
Show search stats in a table
mosharaf13 May 29, 2023
09048a2
[#14] Add search stat show page
mosharaf13 May 29, 2023
80f79ef
[#14] Add list of url's for adword and non-adwords
mosharaf13 May 29, 2023
a8ab938
Add default per page for pagy
mosharaf13 May 29, 2023
6f2fc8a
Use pagy default bootstrap pagination
mosharaf13 May 29, 2023
fb6c01c
Add render to search stats
mosharaf13 May 29, 2023
39da3d2
[#14] Show rendered search stat raw response in modal
mosharaf13 May 30, 2023
d671f14
Merge branch 'backend/list-keywords' into ui/list-keywords
mosharaf13 May 30, 2023
58915f9
[#11] Remove extra blank lines
mosharaf13 May 30, 2023
6e67d79
[#11] Return search stats that belongs to user
mosharaf13 May 31, 2023
4c76675
[#11] Show search stat not found message
mosharaf13 May 31, 2023
65da6d4
Merge branch 'backend/list-keywords' into ui/list-keywords
mosharaf13 May 31, 2023
d57f29e
[#40] Create separate model,migration, and seeder for result links
mosharaf13 Jun 1, 2023
888c0d5
[#40] Show result links from db instead of dummy data
mosharaf13 Jun 1, 2023
08016c1
[#41] Sort models
mosharaf13 Jun 1, 2023
782ca46
Merge branch 'backend/list-keywords' of github.com:mosharaf13/google-…
mosharaf13 Jun 1, 2023
5bd021c
Merge branch 'backend/list-keywords' into ui/list-keywords
mosharaf13 Jun 2, 2023
4a7ee0f
Merge branch 'develop' into ui/list-keywords
mosharaf13 Jun 2, 2023
6a114a7
Merge branch 'ui/list-keywords' of github.com:mosharaf13/google-scrap…
mosharaf13 Jun 6, 2023
ad8a1a8
[#40] Remove ads_page link type from result link model
mosharaf13 Jun 6, 2023
fcafc99
Update db/migrate/20230601085414_create_result_links.rb
mosharaf13 Jun 6, 2023
5af96b7
Update db/seeds.rb
mosharaf13 Jun 6, 2023
81f148c
Update app/views/search_stats/show.html.erb
mosharaf13 Jun 8, 2023
3613756
[#40] Change type of link column in result_links table
mosharaf13 Jun 8, 2023
7517322
Merge pull request #41 from mosharaf13/backend/create-storage-for-res…
mosharaf13 Jun 8, 2023
57f9e8d
[#11] Use active record model attributes in search stats table
mosharaf13 Jun 8, 2023
393df1f
[#11] Add table for search stats
mosharaf13 Jun 8, 2023
a8a2755
[#11] Add active record based translation in search stat show page
mosharaf13 Jun 8, 2023
3a7ee71
[#11] Add search stat title
mosharaf13 Jun 8, 2023
11daa76
[#11] Remove unnecessary code
mosharaf13 Jun 8, 2023
bd74354
[#11] Use js from app/javascript directory to show raw response
mosharaf13 Jun 12, 2023
89990de
[#11] Add relation between search stat and user
mosharaf13 Jun 12, 2023
3746cb9
[#11] Localise search stat details button
mosharaf13 Jun 12, 2023
9159302
[#11] Localise, remove dummy, and fix lint in search stat show page
mosharaf13 Jun 12, 2023
d89551c
[#11] Change link type of result link model
mosharaf13 Jun 12, 2023
fc3e324
[#11] Update config/initializers/pagy.rb
mosharaf13 Jun 12, 2023
8730d24
[#11] Add css :after for search stats details
mosharaf13 Jun 16, 2023
9a8839b
[#11] Fix style
mosharaf13 Jun 16, 2023
f3170c0
[#11] Change style implementation for search stat detail
mosharaf13 Jun 19, 2023
3b2cf62
[#11] Change case for search stat detail class name
mosharaf13 Jun 19, 2023
8079671
[#11] Move raw response modal js file location
mosharaf13 Jun 19, 2023
78e96e5
[#11] Change button element type in search stats table
mosharaf13 Jun 19, 2023
f895479
[#11] Remove wrapping p tag from raw response button
mosharaf13 Jun 19, 2023
055a969
[#11] Localise search stat ads and non ads title
mosharaf13 Jun 19, 2023
51725d5
[#11] Use bootstrap to show raw response modal
mosharaf13 Jun 19, 2023
47dd383
[#11] Localise back button in search stat details page
mosharaf13 Jun 19, 2023
b390c72
[#11] Remove unnecessary js file
mosharaf13 Jun 19, 2023
5bf2a10
[#11] Localise no search stat found message
mosharaf13 Jun 19, 2023
7060001
[#11] Show only top ads in top ads section of search stat details
mosharaf13 Jun 19, 2023
af093c8
[#11] Use single quote for rails string in detail page
mosharaf13 Jun 19, 2023
7ea2920
[#11] Sort en.yml in alphabetical order
mosharaf13 Jun 19, 2023
d4b5083
[#11] Create user explicitly in seeds.rb
mosharaf13 Jun 19, 2023
12b315e
[#11] Sanitize raw response before showing in modal
mosharaf13 Jun 19, 2023
57eeabf
[#11] Update app/assets/stylesheets/application.scss
mosharaf13 Jun 20, 2023
0f869cc
[#11] Update app/controllers/search_stats_controller.rb
mosharaf13 Jun 20, 2023
7c64ab6
[#11] Add indentation in search stat details page
mosharaf13 Jun 22, 2023
e896cb4
[#11] Extract raw response modal iframe style to assets css
mosharaf13 Jun 22, 2023
e11cd8e
[#11] Fix lint
mosharaf13 Jun 22, 2023
4aa79c9
Merge pull request #38 from mosharaf13/ui/list-keywords
mosharaf13 Jun 23, 2023
90247f3
Merge branch 'main' of github.com:mosharaf13/google-scrapper-ruby int…
mosharaf13 Jun 29, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions app/assets/stylesheets/application.scss
Original file line number Diff line number Diff line change
Expand Up @@ -14,5 +14,7 @@
// Layouts

// Components
@import './components/search_stat_details';
@import './components/raw-response-modal';

// Screens
8 changes: 8 additions & 0 deletions app/assets/stylesheets/components/_raw-response-modal.scss
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
.raw-response-modal {
iframe {
width: 100%;
height: 80vh;

border: 0;
}
}
5 changes: 5 additions & 0 deletions app/assets/stylesheets/components/search_stat_details.scss
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
.search-stat-detail {
&__title::after {
content: ':';
}
}
6 changes: 5 additions & 1 deletion app/controllers/search_stats_controller.rb
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,10 @@
class SearchStatsController < ApplicationController
# GET /search_stats
def index
@pagy, @search_stats = pagy(SearchStat.all)
@pagy, @search_stats = pagy(current_user.search_stats)
end

def show
@search_stat = current_user.search_stats.includes(:result_links).find(params[:id])
end
end
9 changes: 9 additions & 0 deletions app/models/result_link.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# frozen_string_literal: true

class ResultLink < ApplicationRecord
enum link_type: { ads_top: 'ads_top', non_ads: 'non_ads' }

belongs_to :search_stat, inverse_of: :result_links

validates :url, presence: true
end
5 changes: 4 additions & 1 deletion app/models/search_stat.rb
Original file line number Diff line number Diff line change
@@ -1,7 +1,10 @@
# frozen_string_literal: true

class SearchStat < ApplicationRecord
validates :keyword, presence: true
has_many :result_links, inverse_of: :search_stat, dependent: :destroy
belongs_to :user, inverse_of: :search_stats

validates :keyword, presence: true, length: { maximum: 255 }
validates :raw_response, presence: true

enum status: { initialized: 0, running: 1, completed: 2, failed: 3 }
Expand Down
2 changes: 2 additions & 0 deletions app/models/user.rb
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ class User < ApplicationRecord
include Authenticable
PASSWORD_PATTERN = /(?=.*[A-Z])(?=.*[a-z])(?=.*\d).{8,}/

has_many :search_stats, inverse_of: :user, dependent: :destroy

before_validation :password_complexity, on: :create

private
Expand Down
14 changes: 14 additions & 0 deletions app/views/search_stats/_raw_response_modal.html.erb
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
<div id="rawResponseModal" class="modal fade raw-response-modal" tabindex="-1" aria-labelledby="rawResponseModalLabel" aria-hidden="true">
<div class="modal-dialog modal-xl">
<div class="modal-content">
<div class="modal-header">
<h5 class="modal-title" id="rawResponseModalLabel">Raw Response</h5>
<button type="button" class="btn-close" data-bs-dismiss="modal" aria-label="Close"></button>
</div>
<div class="modal-body">
<iframe srcdoc="<%= sanitize(@search_stat.raw_response) %>" >
</iframe>
</div>
</div>
</div>
</div>
40 changes: 9 additions & 31 deletions app/views/search_stats/_search_stat.html.erb
Original file line number Diff line number Diff line change
@@ -1,31 +1,9 @@
<div id="<%= dom_id search_stat %>">
<p>
<strong><%= SearchStat.human_attribute_name(:keyword) %>:</strong>
<%= search_stat.keyword %>
</p>

<p>
<strong><%= SearchStat.human_attribute_name(:ad_count) %>:</strong>
<%= search_stat.ad_count %>
</p>

<p>
<strong><%= SearchStat.human_attribute_name(:link_count) %>:</strong>
<%= search_stat.link_count %>
</p>

<p>
<strong><%= SearchStat.human_attribute_name(:total_result_count) %>:</strong>
<%= search_stat.total_result_count %>
</p>

<p>
<strong><%= SearchStat.human_attribute_name(:raw_response) %>:</strong>
<%= search_stat.raw_response %>
</p>

<p>
<strong><%= SearchStat.human_attribute_name(:user_id) %>:</strong>
<%= search_stat.user_id %>
</p>
</div>
<tr id="<%= dom_id search_stat %>">
<td><%= search_stat.keyword %></td>
<td><%= search_stat.ad_count %></td>
<td><%= search_stat.link_count %></td>
<td><%= search_stat.total_result_count %></td>
<td>
<%= link_to t('links.search_stat_details'), search_stat_path(search_stat), class: 'btn btn-primary' %>
</td>
</tr>
31 changes: 26 additions & 5 deletions app/views/search_stats/index.html.erb
Original file line number Diff line number Diff line change
@@ -1,7 +1,28 @@
<h1><%= SearchStat.model_name.human %></h1>
<div class="container mt-4">
<h4><%= SearchStat.model_name.human %></h4>

<div id="search_stats">
<%= render @search_stats %>
</div>
<% if @search_stats.any? %>
<div id="search_stats" class="table-responsive">
<table class="table table-bordered table-striped">
<thead class="thead-light">
<tr>
<th><%= SearchStat.human_attribute_name(:keyword) %></th>
<th><%= SearchStat.human_attribute_name(:ad_count) %></th>
<th><%= SearchStat.human_attribute_name(:link_count) %></th>
<th><%= SearchStat.human_attribute_name(:total_result_count) %></th>
<th></th>
</tr>
</thead>
<tbody>
<%= render @search_stats %>
</tbody>
</table>
</div>

<%== pagy_nav(@pagy) if @pagy.pages > 1 %>
<%== pagy_bootstrap_nav(@pagy) %>
<% else %>
<div class="alert alert-info mt-3">
<p class="mb-0"><%= t('titles.no_stats')%></p>
</div>
<% end %>
</div>
86 changes: 86 additions & 0 deletions app/views/search_stats/show.html.erb
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
<div class="container mt-4">
<div class="d-flex flex-wrap align-items-center">
<div class="mr-auto mb-2">
<%= link_to search_stats_path, class: "btn btn-primary m-1" do %>
<%= t('buttons.back') %>
<% end %>
</div>
</div>

<div class="card mt-4 search-stat-detail">
<div class="card-body">
<h5 class="card-title border-bottom pb-3 mb-4"><%= SearchStat.model_name.human %></h5>
<div class="row">
<div class="col-md-4">
<p class="card-text small">
<strong class="search-stat-detail__title"><%= SearchStat.human_attribute_name(:keyword) %></strong>
<%= @search_stat.keyword %>
</p>

<p class="card-text small">
<strong class="search-stat-detail__title"><%= SearchStat.human_attribute_name(:ad_count_top) %></strong>
<%= @search_stat.top_ad_count %>
</p>

<p class="card-text small">
<strong class="search-stat-detail__title"><%= SearchStat.human_attribute_name(:ad_count) %></strong>
<%= @search_stat.ad_count %>
</p>
</div>
<div class="col-md-4">
<p class="card-text small">
<strong class="search-stat-detail__title"><%= SearchStat.human_attribute_name(:non_ad_count) %></strong>
<%= @search_stat.non_ad_count %>
</p>

<p class="card-text small">
<strong class="search-stat-detail__title"><%= SearchStat.human_attribute_name(:link_count) %></strong>
<%= @search_stat.link_count %>
</p>

<p class="card-text small">
<strong class="search-stat-detail__title"><%= SearchStat.human_attribute_name(:total_result_count) %></strong>
<%= @search_stat.total_result_count %>
</p>
</div>
<div class="col-md-4">
<button type="button" class="btn btn-primary" data-bs-toggle="modal" data-bs-target="#rawResponseModal">
<%= t('buttons.raw_response') %>
</button>
<p class="card-text small">
<strong><%= SearchStat.human_attribute_name(:total_result_count) %>:</strong>
<%= @search_stat.user_id %>
</p>
</div>
</div>
</div>
</div>

<div class="card mt-4">
<div class="card-body">
<h5 class="card-title"><%=t('titles.top_ads')%></h5>
<ul class="list-group">
<% @search_stat.result_links.ads_top.each do |result_link| %>
<li class="list-group-item">
<%= link_to result_link.url, result_link.url %>
</li>
<% end %>
</ul>
</div>
</div>

<div class="card mt-4">
<div class="card-body">
<h5 class="card-title"><%=t('titles.non_ads')%></h5>
<ul class="list-group">
<% @search_stat.result_links.non_ads.each do |result_link| %>
<li class="list-group-item">
<%= link_to result_link.url, result_link.url, target: '_blank' %>
</li>
<% end %>
</ul>
</div>
</div>

<%= render 'raw_response_modal' %>
</div>
9 changes: 9 additions & 0 deletions config/initializers/pagy.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# frozen_string_literal: true

require 'pagy/extras/bootstrap'
require 'pagy/extras/overflow'

# Override default options
Pagy::DEFAULT[:items] = 5
Pagy::DEFAULT[:size] = [1, 2, 2, 1]
Pagy::DEFAULT[:overflow] = :last_page
11 changes: 11 additions & 0 deletions config/locales/en.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,19 @@ en:
attributes:
search_stat:
ad_count: Ad count
ad_count_top: Ad count top
keyword: Keyword
link_count: Link count
non_ad_count: Non ad count
raw_response: Raw response
total_result_count: Total result count
user_id: User
buttons:
back: Back
raw_response: View Raw Response
links:
search_stat_details: Show Details
titles:
no_stats: No search stats found
non_ads: List of URLs of Non-AdWords Results on the Page
top_ads: List of URLs of AdWords Advertisers in Top Position
2 changes: 1 addition & 1 deletion config/routes.rb
Original file line number Diff line number Diff line change
Expand Up @@ -8,5 +8,5 @@
root "home#index"

get "/health_check", to: 'health_check#health_check', as: :rails_health_check
resources :search_stats, only: [:index]
resources :search_stats, only: [:index, :show]
end
13 changes: 13 additions & 0 deletions db/migrate/20230601085414_create_result_links.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
class CreateResultLinks < ActiveRecord::Migration[7.0]
def change
enable_extension 'citext' unless extension_enabled?('citext')

create_table :result_links do |t|
t.references :search_stat, null: false, foreign_key: true
t.integer :link_type, null: false
t.citext :url, null: false

t.timestamps
end
end
end
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
class ChangeUrlTypeInResultLinks < ActiveRecord::Migration[7.0]
def change
remove_index :result_links, :url

change_column :result_links, :url, :string, null: false

add_index :result_links, :url
end
end
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
class ChangeLinkTypeToStringInResultLinks < ActiveRecord::Migration[7.0]
def change
change_column :result_links, :link_type, :string
end
end
43 changes: 42 additions & 1 deletion db/schema.rb

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

9 changes: 7 additions & 2 deletions db/seeds.rb
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,11 @@
require 'fabrication'

# Generate dummy data for SearchStat
100.times do
Fabricate(:search_stat)

user = User.find_or_initialize_by(email: 'user@demo.com') do |user|
user.password = 'aaaaaaA1'
end

10.times do
Fabricate.times(100, :search_stat, user: user)
end
10 changes: 10 additions & 0 deletions spec/fabricators/result_link_fabricator.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# frozen_string_literal: true

Fabricator(:result_link) do
link_type { ResultLink.link_types.keys.sample }
url { FFaker::Internet.http_url }
end

Fabricator(:result_link_with_search_stat, from: :result_link) do
search_stat { Fabricate(:search_stat) }
end