Central Web Services

Access.cws will be moved to a new location, this will begin at Saturday the 26th at 10pm. I’m only expecting a few minutes of downtime but am allowing for an hour to deal with anything that may come up. Please close any connections (SSH/SFTP/Samba) prior to the outage to avoid any possible loss of data.

The following services will not be available during the outage: *The ability to update a website via SSH/SFTP/Samba on access.cws.oreognstate.edu, this will not affect the ability to update a Drupal site. *The website http://images.ecampus.oregonstate.edu

Banner and associated systems will be out of service beginning at 11:55 pm on Friday, May 25th until 5:00 pm Saturday, May 26th. The associated systems include:

• Appworx
• Banner
• EvalS
• MyDegrees
• myOSU
• Resource25
• Touchnet (including Payment Gateway, Marketplace, and Cashiering)
• Workflow

Catherine Williams Associate Director, Enterprise Computing Services Oregon State University 224A Milne Computer Center Corvallis OR 97331 541-737-2718

We were unable to get the Brocade load balancers to pass NTLM authentication so we reverted back to the original servers. NTLM authentication is required for Outlook Anywhere (RPC over HTTP). We will be taking this issue up with Brocade next week to see if they can provide the configuration changes required to make this work. However, the rest of our tests were successful so there should be less of a service disruption the next time this change is attempted.

Cary

(My working hours are 7am – 5:30 pm, Tuesday – Friday) ———————————————————- Cary Shufelt, MCSA, MCSE Windows Infrastructure Architect Network Engineering B211 Kerr Admin Bldg Oregon State University Corvallis, OR 97331 541-737-9420

The OSU home page was unavailable between 10:17 and 10:22 am, 5/18.  We experienced a failure of the xen pool master which houses our primary nfs server. This server mounts our homepage and web root.

We have brought up another nfs server while we troubleshoot the non responsive pool master.

This maintenance has been completed.

On Tue, 15 May 2012, net@oregonstate.edu wrote:

> Date: Tue, 15 May 2012 09:25:05 -0700 > From: net@oregonstate.edu > To: maintenance@lists.oregonstate.edu > Subject: [Maintenance] Root Bridget Maintenance > Resent-Date: Tue, 15 May 2012 09:26:02 -0700 (PDT) > Resent-From: > > From: Joel Burks > > Service(s) affected: These vlans could be potentially affected. > > VLAN0405 IOSCS_SdmzL > VLAN0415 ICoreNet > VLAN0417 ISAD_Hwebcam > VLAN0465 IISsvcs_SVMotion > VLAN0479 ICWS_HVMMgt > VLAN0484 InCGRBsvcs > VLAN0486 ISCF_WAdmin > VLAN0488 ISCF_SMedia > VLAN0496 InServ02svcs > VLAN0602 Network-Security-DMZ > VLAN0622 TermS > VLAN0777 InUHDS-EMC > VLAN1522 OWirelessAccessPool4 > VLAN1525 OutWired_OSU1 > VLAN1526 OWiFi_Phone1 > VLAN1533 OWirelessAccessPool3 > VLAN1534 OWirelessAccessPool1 > VLAN1535 OWirelessAccessPool2 > VLAN1620 OutEnterprise-VPN > VLAN0014 Misc:MISC-AG > VLAN0256 ResNet:256 > VLAN0257 ResNet:257 > VLAN0258 ResNet:258 > VLAN0259 ResNet:259 > VLAN0260 ResNet:260 > VLAN0261 ResNet:261 > VLAN0262 ResNet:262 > VLAN0263 ResNet:263 > VLAN0401 InTELsvcs > VLAN0402 InNET_NLB > VLAN0418 IMCS_Hvideo > VLAN0466 ICOEsvcs_Hwireless > VLAN0493 InNetmon > VLAN0495 InSHSsvcs > VLAN0498 InNetsvcs > VLAN0514 exch > VLAN0521 OSUFND_Wireless-FW > VLAN0522 IWirelessAccessPool4 > VLAN0526 IWiFi_Phone1 > VLAN0532 Wireless-Aruba > VLAN0533 IWirelessAccessPool3 > VLAN0534 IWirelessAccessPool1 > VLAN0535 IWirelessAccessPool2 > VLAN0537 WirelessSecurePool2 > VLAN0538 WirelessSecurePool3 > VLAN0539 WirelessSecurePool4 > VLAN0732 WinDCIN > VLAN0733 nwsWININ > VLAN0815 VM-ESX-Mngt > VLAN0821 Athletics-non-Wks > VLAN0003 IS:test3 > VLAN0505 WAP_Management2 > VLAN0509 SHS-FW > VLAN0513 OCH-Quarantine > VLAN0536 WirelessSecurePool1 > VLAN0554 COE-Backup > VLAN0621 OSUVPN > VLAN0624 OUS-OSU > VLAN0626 NWREC_Aurora > VLAN0701 InKES > VLAN0704 IFWL-FW_WWkstn1 > VLAN0705 IFWL-FW_WLegacy > VLAN0712 ICOS_FW_Wkstn1 > VLAN0776 ICOB_WLab > VLAN0777 InUHDS-EMC > VLAN0778 ICOB_WFS > VLAN0779 ICOB_HBSG > VLAN0807 Hort-workstations > VLAN0808 Athletics-workstations > VLAN0811 Registration-Enrollment > VLAN0814 IS-Media-FW > VLAN0818 HPBuilding11 > VLAN0820 AgResFnd > VLAN0821 Athletics-non-Wks > VLAN0830 FlexNet:BAF-FW > VLAN0840 COE-FW > VLAN0865 CWS-wkst > VLAN0866 SFlexNet > VLAN0875 UHDS-FW > VLAN0876 UHDS-SWM > VLAN0882 CSS-FW > VLAN0884 CGRB-FW > > > The root bridge for these vlans will be moved to the device routing the specific vlan. > > Maintenance Window: > Start: 05/17/2012 6:30 AM > End: 05/17/2012 7:15 AM > > Reason for maintenance: > Maintaining spanning-tree protocol best practices. > > Impact: > > Departments affected: Any department that uses services on the above mentioned vlans.I have given a short description of the purpose of the vlans to help understand the potential impact. > > Locations affected: NA > > Applications affected: Any service that uses the above mentioned vlans.I have given a short description of the purpose of the vlans to help understand the potential impact. > > If you have questions or concerns about this maintenance, please contact Network Engineering at net (at) oregonstate.edu or call 737-HELP. >

There is an architectural change in Exchange 2010 that moves Outlook MAPI mailbox connections from the back-end mailbox servers to the Client Access servers (CAS). Providing both a load balanced and highly available Client Access Server (CAS) solution is even more important than was the case with our current version of Exchange so we are introducing a pair of hardware load balancers into our Exchange infrastructure.

This change requires that we move the DNS A record for EXMAIL.oregonstate.edu from the Windows load balanced cluster to a pair of Brocade ADX 1000s. The following Exchange services may be affected by this move:

Outlook Web App (OWA) Exchange ActiveSync (EAS) Outlook Anywhere (RPC over HTTP) Exchange Web Services (EWS) – Synchronization, Notification, Availability, and Automatic Replies (OOF) IMAP and POP3 Autodiscover RPC Client Access Service (RPC CA) Exchange Address Book Service

Of these services, we have only been able to fully test OWA, IMAP & POP3 using a local hosts file. All other client services receive the IP address of the service from the Autodiscover service so we were unable to test their functionality prior to this change. On Friday evening we will be making the DNS change so we can move forward with our Exchange 2010 deployment. Once this change is made we will test and verify the functionality of all services provided by EXMAIL. We have set the TTL of the A record to 5 min so we can move the service back to the original servers if our tests do not go well. In the last EXMAIL move, we found that many clients (particularly, MACs and mobile devices) did not honor the TTL and instead retained the entries in their local DNS cache for 24 hours. We have chosen Friday evening to make this change so clients that pick up the DNS change will have time to renew their cached DNS entries by Monday morning if we need to revert back to the original servers.

Anticipated Timeline: 5/18/2012, 10:00 pm – Initiate A record move 5/18/2012, 10:15 pm – A record move complete 5/18/2012, 10:15 – 10:30 pm – Verify DNS changes on servers and test clients. 5/18/2012, 10:30 – 11:59 pm – perform 1st phase of tests. I will be performing a battery of tests against each of 3 physical servers. If these tests fail then the DNS change will be reversed changes and services verified. If the tests pass then all 3 physical servers will be activated in the load balanced array. A notice will be sent to DCA-ALL and Outages indicating the status of the deployment. 5/19/2012, 12:00 am – 5:00 pm. Additional testing on a wide variety of client devices by CN staff the DCA community.* 5/19/2012, 5:00 pm – Review the results of the tests and decide whether to move forward with the load balancer deployment or to reverse the changes.

If we decide to move the load balancer into production, the existing Windows load balanced servers will be kept online for a week. This will allow clients that do not pick up the DNS change to continue to connect and be able to send & receive email. We anticipate that there will be some client impact during this process so customers should be informed that the OWA service will be available in case they cannot connect. All reports of service unavailability should be forwarded to itconsult@oregonstate.edu and include the following information:* Username Connection type (Active Sync, IMAP, POP3, Outlook, Outlook Anywhere, EWS, etc.) IP address (if known) Software client version (Outlook 2007, 2010, 2011, Mac Mail, Entourage, etc.) Time of connection attempt

* Testers and customers reporting issues should flush their DNS to ensure that they have correct DNS information. The IP address of the hardware load balanced EXMAIL.oregonstate.edu is 128.193.7.79, the existing Windows load balance array is: 128.193.232.48. The commands to flush the DNS cache are: PC : Close all open web browsers and applications on your computer Open the Windows command prompt by selecting the “Start” menu and entering “cmd” in the search text field box. Enter the command: ipconfig /flushdns

Mac: If Mozilla Firefox is installed on your computer, exit the application if it is open. Open the terminal on your computer. On a computer running Lion (Mac OS X 10.5, 10.6, or 10.7) enter the following command followed by pressing the “return” key: dscacheutil -flushcache In Mac OS X 10.4 Tiger, enter the following command followed by pressing the “return” key: lookupd -flushcache

On mobile devices it varies but power cycling or toggling airplane mode has been reported to work.

Cary

(My working hours are 7am – 5:30 pm, Tuesday – Friday) ———————————————————- Cary Shufelt, MCSA, MCSE Windows Infrastructure Architect Network Engineering B211 Kerr Admin Bldg Oregon State University Corvallis, OR 97331 541-737-9420

Cary

(My working hours are 7am – 5:30 pm, Tuesday – Friday) ———————————————————- Cary Shufelt, MCSA, MCSE Windows Infrastructure Architect Network Engineering B211 Kerr Admin Bldg Oregon State University Corvallis, OR 97331 541-737-9420

Notice of Unplanned Outage

This morning we experienced high load on our border router as a result of this mornings attempt to remove NAT from the border router. This caused an interruption in service for several remote phone and data networking sites between 7 and 8:30 this morning until NAT was reintroduced at the border router. We are still investigating why this change resulted in high load on the router.

If you have questions or concerns about this outage, please contact Network Engineering at net (at) oregonstate.edu or call 737-HELP.

Service(s) affected: mailman

We are moving the port on lists.oregonstate.edu to the new nexus gear. This move should take less than 5 minutes and we should not see any outage at all, if there is a problem mail will be queued up at the relays so loss of mail is not expected

Maintenance Window: Start: 05/17/2012 7:00 AM End: 05/17/2012 7:45 AM

Reason for maintenance: Moving ports to new nexus gear

Impact:

Applications affected: mailman

If you have questions or concerns about this maintenance, please contact Network Engineering at net (at) oregonstate.edu or call 737-HELP.

Service(s) affected: These vlans could be potentially affected.

VLAN0405 IOSCS_SdmzL VLAN0415 ICoreNet VLAN0417 ISAD_Hwebcam VLAN0465 IISsvcs_SVMotion VLAN0479 ICWS_HVMMgt VLAN0484 InCGRBsvcs VLAN0486 ISCF_WAdmin VLAN0488 ISCF_SMedia VLAN0496 InServ02svcs VLAN0602 Network-Security-DMZ VLAN0622 TermS VLAN0777 InUHDS-EMC VLAN1522 OWirelessAccessPool4 VLAN1525 OutWired_OSU1 VLAN1526 OWiFi_Phone1 VLAN1533 OWirelessAccessPool3 VLAN1534 OWirelessAccessPool1 VLAN1535 OWirelessAccessPool2 VLAN1620 OutEnterprise-VPN VLAN0014 Misc:MISC-AG VLAN0256 ResNet:256 VLAN0257 ResNet:257 VLAN0258 ResNet:258 VLAN0259 ResNet:259 VLAN0260 ResNet:260 VLAN0261 ResNet:261 VLAN0262 ResNet:262 VLAN0263 ResNet:263 VLAN0401 InTELsvcs VLAN0402 InNET_NLB VLAN0418 IMCS_Hvideo VLAN0466 ICOEsvcs_Hwireless VLAN0493 InNetmon VLAN0495 InSHSsvcs VLAN0498 InNetsvcs VLAN0514 exch VLAN0521 OSUFND_Wireless-FW VLAN0522 IWirelessAccessPool4 VLAN0526 IWiFi_Phone1 VLAN0532 Wireless-Aruba VLAN0533 IWirelessAccessPool3 VLAN0534 IWirelessAccessPool1 VLAN0535 IWirelessAccessPool2 VLAN0537 WirelessSecurePool2 VLAN0538 WirelessSecurePool3 VLAN0539 WirelessSecurePool4 VLAN0732 WinDCIN VLAN0733 nwsWININ VLAN0815 VM-ESX-Mngt VLAN0821 Athletics-non-Wks VLAN0003 IS:test3 VLAN0505 WAP_Management2 VLAN0509 SHS-FW VLAN0513 OCH-Quarantine VLAN0536 WirelessSecurePool1 VLAN0554 COE-Backup VLAN0621 OSUVPN VLAN0624 OUS-OSU VLAN0626 NWREC_Aurora VLAN0701 InKES VLAN0704 IFWL-FW_WWkstn1 VLAN0705 IFWL-FW_WLegacy VLAN0712 ICOS_FW_Wkstn1 VLAN0776 ICOB_WLab VLAN0777 InUHDS-EMC VLAN0778 ICOB_WFS VLAN0779 ICOB_HBSG VLAN0807 Hort-workstations VLAN0808 Athletics-workstations VLAN0811 Registration-Enrollment VLAN0814 IS-Media-FW VLAN0818 HPBuilding11 VLAN0820 AgResFnd VLAN0821 Athletics-non-Wks VLAN0830 FlexNet:BAF-FW VLAN0840 COE-FW VLAN0865 CWS-wkst VLAN0866 SFlexNet VLAN0875 UHDS-FW VLAN0876 UHDS-SWM VLAN0882 CSS-FW VLAN0884 CGRB-FW

The root bridge for these vlans will be moved to the device routing the specific vlan.

Maintenance Window: Start: 05/17/2012 6:30 AM End: 05/17/2012 7:15 AM

Reason for maintenance: Maintaining spanning-tree protocol best practices.

Impact:

Departments affected: Any department that uses services on the above mentioned vlans.I have given a short description of the purpose of the vlans to help understand the potential impact.

Locations affected: NA

Applications affected: Any service that uses the above mentioned vlans.I have given a short description of the purpose of the vlans to help understand the potential impact.

If you have questions or concerns about this maintenance, please contact Network Engineering at net (at) oregonstate.edu or call 737-HELP.

Service(s) affected: Workstations with host addresses other than 128.193.x.y

I will be turning off Network Address Translation at the border router for subnets that have had their NAT moved to a firewall.

Maintenance Window: Start: 05/16/2012 6:30 AM End: 05/16/2012 7:00 AM

Reason for maintenance: NAT for these subnets has been moved to their respective firewalls so NAT at the border router is no longer needed.

Impact:

Departments affected: Departments previously notified of moving nat to their firewall

Locations affected: Various

Applications affected: Any application on above workstations attempting to access off-campus sites.

If you have questions or concerns about this maintenance, please contact Network Engineering at net (at) oregonstate.edu or call 737-HELP.

Service(s) affected: All vlans on campus can potentially be affected, which would affect most services on campus.

VTP will be enabled on the data center switches and joined to the core network’s VTP domain in client mode.

Maintenance Window: Start: 05/19/2012 10:00 PM End: 05/19/2012 11:45 PM

Reason for maintenance: This will allow vlans to be more easily managed across the core network.

Impact:

Departments affected: This change should not affect the network. But it does have the potential for affecting all of campus departments.

Applications affected: Applications affected: This change should not affect the network. But it does have the potential for affecting all of campus applications.

If you have questions or concerns about this maintenance, please contact Network Engineering at net (at) oregonstate.edu or call 737-HELP.

Service(s) affected: All vlans on campus can potentially be affected, which would affect most services on campus.

VTP will be enabled on the data center switches and joined to the core network’s VTP domain in client mode.

Maintenance Window: Start: 05/12/2012 10:00 PM End: 05/12/2012 11:45 PM

Reason for maintenance: This will allow vlans to be more easily managed across the core network.

Impact:

Departments affected: This change should not affect the network. But it does have the potential for affecting all of campus departments.

Locations affected: NA

Applications affected: This change should not affect the network. But it does have the potential for affecting all of campus applications.

If you have questions or concerns about this maintenance, please contact Network Engineering at net (at) oregonstate.edu or call 737-HELP.

Notice of Unplanned Outage

This morning may have experienced a brief set of outages on the wireless network between 8AM and 8:45AM when what appears to be a large number of packets affected performance on the the wireless router. We are still investigating the cause. Please let us know if you encountered issues during this period.

If you have questions or concerns about this outage, please contact Network Engineering at net (at) oregonstate.edu or call 737-HELP.

The systems listed below will be out of service from 7:00 am until noon on Saturday, May 5th.

• AppWorx
• MyDegrees
• MyOSU Portal
• EvalS
• Resource25
• Touchnet (including Payment Gateway, Marketplace and Cashiering)
• Workflow

Parts or all of OSU experienced poor Internet connectivity on the afternoon of 5/1 between 3:45 and 4:05 pm.  The issue should be resolved now.  There was a DDoS against a node in Corvallis, outside of the OSU network, that impacted traffic for others.

On Tuesday May 1st Central Web Services will be upgrading hosted production sites to the latest release of OSU Drupal. This release includes a some important security updates, as well as accessibility upgrades to the OSU Confident theme, and an update to the theme for College of Agricultural Sciences.

Please read the announcement for full details of what is included in this release.

The schedule for updating your sites is as follows:

• Tuesday May 1st 8:00 AM to 5:00 PM – Update production sites.
• Wednesday May 2nd 8:00 AM to 12:00 PM – Update group and organization (VSO and SSO) production sites.

This is a rolling update and we cannot say specifically when each site will be updated. The update takes about 5 minutes for each site during which a visitor to the site may see the OSU Drupal maintenance screen displaying your site maintenance message. You can view or change your site maintenance message by navigating to Site Configuration -> Site Maintenance when logged in as a site administrator.

Please contact us if you have any questions or concerns.

– Paul Lieberman Lead Web Developer Central Web Services

Comcast Cable lent OSU a transmitter and we are now back on-line.

Sent: Wednesday, April 25, 2012 9:49 AM

The equipment used to provide TV feeds from Media Services to the Campus COMCAST nodes is having intermittent issues. Degraded services include MTV-u, Oregon WIN feeds. (channels 14-18) I am trying to locate a replacement transmitter.

Thomas E Williams Chief Engineer Oregon WIN www.oregonwin.org Transmission Engineer, Oregon State University http://oregonstate.edu 541-737-3822

Service(s) affected: Wireless OSU Access

Authentication to OSU access will be unavailable for short periods of time durring the one hour window. Users already authenticated will not see an issue, same with those using OSU Secure.

Outage Window: Start: 05/02/2013 6:30 AM End: 05/02/2013 7:30 AM

Reason for outage: We are continuing to troubleshoot wireless issues. As part of the troubleshooting we need to interrupt service to OSU Access’s authentication.

Impact:

Departments affected: All

Locations affected: All

Applications affected: OSU Access

Network Outage Policy http://oregonstate.edu/net/info/policy/outagepolicy.php

If you have questions or concerns about this outage, please contact Network Engineering at net (at) oregonstate.edu or call 737-HELP.

Service(s) affected: Here is a list of servers connected to a016nxs

nws-mssql2a-1 del nws-mssql1a-1 del nws-mssql1b-1 del nws-mssql3a-1 del nws-mssql3b-1 del nws-hts-1 dell01- nws-dc3.nws.orego dc3.oregonstate.e nws-sp1a dell07-2 lync-fe1a dell07- S8730b mcs-mgc; osu3-00 mcs-mgc1; osu3-0 mcs-mgc2; osu3-0 mcs-rmx1e Netbay5 mcs-dpipc-c1 Netb 1 mcs-dpipc-r1 Netb mcs-dpipc-sem1 Ne mcs-tms1 Netbay5 mcs-renovo1 Netba mcs-cma1 Netbay5 mcs-dma1 Netbay5 mcs-rmx1a Netbay5 mcs-rmx1b Netbay5 mcs-rmx1c Netbay5 mcs-rmx1d Netbay5 argosjr; Dell3-25 KAdm-A016-NetBay1 mcs-tcr1 NetBay7 mcs-mcu6 NetBay7 mcs-gk1 NetBay7 D mcs-pdu1 NetBay5 Outside VoIP test Inside VoIP test P nws-ca1b dell01-0 12A02-MedRes Tele 01B10-CLAN TeleBa 01A03-MedRes Tele nws-mssql2a-2 del nws-mssql1a-2 del nws-mssql1b-2 del nws-mssql3a-2 del nws-mssql3b-2 del nws-sp1b dell07-2 lync-fe1b dell07- ex1.oregonstate. ex1.oregonstate.e nws-mssql2b dell6 smtp7 Dell5-8 lync-gw1 dell07-3 mcs-rmx2e Netbay6 mcs-dpipc-c2 Netb 1 mcs-dpipc-r2 Netb mcs-dpipc-sem2 Ne mcs-tms2 Netbay6 mcs-renovo2 Netba mcs-cma2 Netbay6 mcs-dma2 Netbay6 mcs-rmx2a Netbay6 mcs-rmx2b Netbay6 mcs-rmx2c Netbay6 mcs-rmx2d Netbay6 mcs-pdu2 NetBay6 fesapp1-mgmt; Del 2 mcs-pv1 mcs-pv2 aruba-3600-M; Net aruba-3600-B; Net aruba-3600-L1; Ne aruba-3600-L2; Ne aruba-3600-L3; Ne aruba-3600-L4; Ne aruba-3600-L5; Ne aruba-3600-L13; N zebi-kvm.nws.oreg fiberswitch1.nws fiberswitch2.nws vm-zebi.nws.orego mcu1a netbay7 D.0 mcu1b netbay7 D.0 mcu1c netbay6 D.0 mcu1d netbay6 D.0

The port channel from a016nxs to drt1 will flap, or go down and back up quickly. The actual downtime should be less than a minute.

Outage Window: Start: 04/28/2012 10:00 PM End: 04/28/2012 10:15 PM

Reason for outage: The vpc domain id needs changed to allow connectivity to the data center switches.

Impact:

Departments affected: NA

Locations affected: NA

Applications affected: NA

Network Outage Policy http://oregonstate.edu/net/info/policy/outagepolicy.php

If you have questions or concerns about this outage, please contact Network Engineering at net (at) oregonstate.edu or call 737-HELP.

The equipment used to provide TV feeds from Media Services to the Campus COMCAST nodes is having intermittent issues. Degraded services include MTV-u, Oregon WIN feeds (channels 14-18). We are working to supply a replacement transmitter.